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PREFACE 


This  publication  documents  a  briefing  presented  to  the  Honorable  Gilbert  F.  Decker,  Assistant 
Secretary  of  the  Army  (Research,  Development,  and  Acquisition),  Dr.  Kenneth  J.  Oscar,  Deputy 
Assistant  Secretary  of  the  Army  (Procurement),  and  Mr.  Dale  Adams,  Principal  Deputy  for 
Acquisition,  U.S.  Army  Materiel  Command,  on  Novembers-?,  1997.  It  describes  research 
conducted  as  part  of  a  project  entitled  “Understanding  Incentives  in  the  Army  Acquisition  System.” 
This  briefing  focuses  on  work  force  perspectives  on  acquisition  reform  and  recommends  several 
policy  options  for  improving  attitudes,  beliefs,  and  behaviors  in  support  of  the  reform  effort. 

The  research  was  sponsored  by  the  Assistant  Secretary  of  the  Army  (Research,  Development, 
and  Acquisition)  and  U.S.  Army  Materiel  Command  and  was  conducted  in  the  Force  Development 
and  Technology  Program  of  RAND’s  Arroyo  Center.  The  Arroyo  Center  is  a  federally  funded 
research  and  development  center  sponsored  by  the  United  States  Army. 
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As  noted  in  the  preface,  this  briefing  documents  the  findings  of  a  study  of  “incentives”  in  the 
Army  acquisition  system.  The  study  was  sponsored  by  the  Assistant  Secretary  of  the  Army 
for  Research,  Development,  and  Acquisition  (SARDA)  and  the  Army  Materiel  Command’s 
Office  of  the  Principal  Deputy  for  Acquisition. 

In  1 994  the  Department  of  Defense  (DoD)  began  to  implement  acquisition  reform  as 
outlined  in  the  “Perry  Initiatives.”  These  initiatives  were  designed  to  increase  the  flexibility 
and  speed  of  DoD’s  acquisition  system  while  simultaneously  reducing  the  costs  inherent  in 
contracting  with  the  military  services.  These  initiatives  included  three  major  policy  thrusts: 

•  eliminate  use  of  military  specifications  (milspecs)  and  standards,  and  replace  them 
with  performance-based  specifications, 

•  expand  use  of  integrated  product/process  teams  (IPTs)  in  acquisition,  and 

•  develop  “partnerships” — in  line  with  those  used  in  the  private  sector — with  major  DoD 
suppliers. 

The  Department  of  the  Army  embraced  these  reforms  and  began  efforts  to  implement  them 
throughout  its  acquisition  system.  However,  after  initial  attempts  to  implement  these 
policies.  Army  leadership  began  to  be  concerned  that  the  reforms  were  not  being 
embraced  by  the  “rank  and  file”  of  the  acquisition  work  force. 

This  study  examines  support  for  the  various  reforms  within  the  Army  acquisition  work  force 
and  suggests  changes  to  current  implementation  and  training  efforts.  While  initially 
designed  to  examine  the  role  of  personnel  policies  and  incentives  in  motivating 
compliance,  the  study  evolved  into  a  broader  examination  of  the  role  of  employee  attitudes, 
perception,  and  organizational  support  mechanisms  in  the  process  of  acquisition  reform. 
What  we  have  learned  in  this  effort  is  described  in  the  following  charts. 
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The  figure  in  this  chart  describes  the  conceptual  framework  that  guided  our  research.  This 
figure  represents  our  conception  of  the  Army’s  reform  effort  and  the  factors  important  in 
determining  its  effectiveness.  Essentially,  this  figure  is  our  model  of  the  changes  taking 
place  in  Army  acquisition.  It  is  based  on  observation  of  Army  reform  efforts  and  input  from 
literature  on  organizational  change/management,  “reinventing”  government,  and 
bureaucratic  behavior.  Each  element  of  the  chart  is  described  below. 

Strategic  goals:  The  overall  goals  of  the  “Perry  Initiatives”  provide  the  context  within  which 
the  Army’s  reform  efforts  are  taking  place.  The  strategic  goals  of  reform  aim  to  create  an 
acquisition  system  that  is  more  flexible,  integrated,  and  less  “stovepiped”;  that  exhibits 
elements  of  “best  practice”  in  procurement  and  supplier  relations;  and  that  is  based  on 
cooperative  relations  with  suppliers. 

Major  initiatives:  As  noted  earlier,  there  are  three  major  initiatives  designed  to  help  meet 
strategic  goals  and  produce  effective  reform  (a  more  commercial,  faster,  better,  and 
cheaper  acquisition  system)— milspec  and  standard  reform,  IPTs,  and  supplier 
partnerships.  While  often  described  independently,  the  initiatives  are  interrelated  and 
mutually  supportive.  As  will  be  described  in  more  detail  in  subsequent  charts,  efforts  to 
eliminate  milspecs  and  standards  and  move  toward  partnerships  with  industry  are  affected 
by  the  success  of  IPTs.  To  the  extent  that  IPTs  bridge  functional  gaps  in  the  management 
process,  facilitate  information  flows,  and  reduce  risk  among  stakeholders,  they  make 
change — such  as  operating  without  traditional  milspecs  and  standards— easier  for  Army 
personnel.  In  addition,  IPTs  are  an  important  element  in  effective  supplier  partnerships. 
While  IPTs  are  encouraged  for  use  internally  within  the  Army  to  improve  management,  they 
are  also  proposed  externally— between  the  Army  and  its  contractors— to  improve 
communication  between  the  Army  and  the  weapon  producer/supplier.  Thus,  effective  IPTs 
also  facilitate  the  movement  toward  better  supplier  relations.  While  support  for  each  of 
these  initiatives  will  be  examined  individually,  it  is  important  for  the  reader  to  keep  in  mind 
that  they  are  indeed  related. 
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Mediating  factors:  Whiie  the  reform  initiatives  are  designed  to  produce  desired  changes 
within  the  Army  acquisition  system,  their  true  effect  or  success  wiii  be  determined  by 
mediating  factors  within  the  work  force  and  the  organization.  These  factors  are  drawn  from 
the  literature  on  bureaucratic  behavior  and  organizationai  change/management .  Basicaliy, 
mediating  factors  can  be  broken  into  two  categories:  empioyee  attitudes  and  organizationai 
support  mechanisms. 

Ciearly,  the  behavior  of  public  employees — such  as  support  of  reforms — will  depend  on 
their  assessment  of  factors  such  as  beiiefs  about  how  reform  will  affect  them  personally 
(personal  beliefs),  their  view  of  co-workers  and  professional  standards  (professional 
norms),  and  beliefs  about  how  reform  will  affect  their  organizational/work  objectives 
(programmatic  beliefs). 

Organizational  change  literature  notes  that  specific  organizational  attributes  can  constrain 
or  facilitate  the  adoption  of  reform  in  an  organization.  These  factors  are  identified  on  the 
chart,  and  include  organizational  structure  and  human  resource  management  practices, 
training,  employee  empowerment,  communication,  and  information  systems.  The 
adequacy  (or  the  perceived  adequacy)  of  each  of  these  mechanisms  and  their  use  in 
change  efforts  can  be  important  in  producing  effective  reform. 

Effective  reform  and  organizational  change:  The  reform  initiatives,  filtered  through  the 
mediating  factors,  are  designed  to  produce  effective  reform — essentially,  meeting  strategic 
objectives.  It  Is  the  belief  of  DoD  and  Army  leadership  that  an  acquisition  reform  system 
based  on  performance  specifications,  internal  and  external  IPTs,  and  improved  buyer- 
supplier  relations  will  result  in  better,  less-expensive  products  that  are  produced  in  a  more 
timely  fashion. 

Will  these  three  reforms  result  In  the  desired  changes?  This  study  does  not  address  this 
question.  It  focuses  exclusively  on  the  implementation  of  the  reform  initiatives,  be  they 
advisable  or  not  (it  is  generally  our  belief  that  they  are  advisable).  Having  said  this,  it  is 
evident  what  this  study  does  not  do.  This  research  does  not  evaluate  the  effect  of  reform 
policies  or  their  adequacy  in  meeting  the  strategic  goals  of  reform.  Instead  it  examines  the 
potential  success  of  current  reform  efforts  by  examining  the  extent  to  which  employee 
attitudes  and  support  mechanisms  are  conducive  to  effective  change.  In  other  words,  this 
study  will  examine  the  various  reforms,  the  state  of  their  implementation,  and  the  extent  to 
which  the  mediating  factors  appear  to  facilitate  or  constrain  effective  change.  It  is  hoped 
that  the  observations  and  conclusions  made  here  about  attitudes  and  support  mechanisms 
will  help  Army  policymakers  improve  and  target  their  implementation  efforts.  To  achieve 
this  goal,  the  study  asked  a  series  of  key  research  questions,  identified  on  the  next  slide. 
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Research  Questions 

•  What  are  the  prevailing  attitudes  toward  reform? 

•  In  what  ways  are  attitudes  and  beliefs  affecting 
behaviors? 

•  How  do  organizational  support  mechanisms  affect 
support  and  behavior? 

•  How  can  current  implementation  efforts  be 
improved? 

Arroyo  center  ba 


In  conducting  our  work,  we  attempted  to  address  the  following  four  key  research  questions. 
Each  question  is  designed  to  provide  information  on  the  state  of  current  implementation 
efforts  and  the  factors  affecting  employee  willingness  to  adopt  the  reforms. 

What  are  the  prevailing  attitudes  toward  reform?  This  question  is  essential  in  the  initial 
assessment  of  support  for  each  of  the  reform  initiatives.  Obviously,  support  levels  can  vary 
for  the  reforms  individually  or  among  groups  within  the  acquisition  work  force.  Determining 
basic  support  levels  and  the  extent  to  which  the  reforms  have  been  adopted  is  essential 
information  for  policymakers. 

In  what  ways  are  attitudes  and  beliefs  affecting  behaviors?  Understanding  the  effect 
of  employee  attitudes  on  behavior  will  be  essential  information  if  suggestions  are  to  be 
made  on  how  to  restructure  training  and  implementation  policies.  In  order  to  correct  beliefs 
and  attitudes  that  are  damaging  to  reform  efforts  or  to  leverage  beliefs  and  attitudes  that 
are  supportive  of  reform  efforts,  policymakers  must  understand  how  employee  attitudes 
and  beliefs  affect  behavior. 

How  do  organizational  support  mechanisms  affect  support  and  behavior?  This 
question  examines  the  extent  to  which  organizational  factors  affect  employee  attitudes  and 
support  levels.  Identifying  the  effect  of  these  factors  on  support  and  behavior  may  identify 
viable  “policy  levers”  for  consideration  by  Army  management. 

How  can  current  implementation  efforts  be  improved?  This  is  the  primary  question 
pursued  by  this  study.  Having  identified  the  role  of  attitudes  and  organizational  support 
mechanisms  on  employee  support  and  behavior,  it  should  be  possible  to  provide  general 
guidelines  to  Army  officials  on  how  to  better  target  their  reform  implementation  and  training 
efforts. 
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Research  Methods 


Detailed  case  studies  of  programs  and  matched  contractors 

•  Army  programs:  ATACMS-BAT,  Longbow,  PLS,  M-1  Abrams, 
Bradley  IFV,  PM-night  vision,  PM-SATCOM,  Advanced  towed 
cannon,  and  FSCATTS 

•  Contractors:  GD,  Hughes,  Tl,  MD,  Lockheed-Martin,  Litton,  Loral, 
GTE,  Oshkosh,  and  United  Defense 

On-site  field  interviews  to  analyze  large-scale  change  efforts  in  the 
private  and  public  sectors 

•  Chrysler,  NYNEX,  Xerox,  Tl,  a  large  conglomerate,  and  the  state 
of  Washington 

Survey  of  the  Army  acquisition  work  force 

•  Stratified  random  sample  of  3,000  DAWIA  designated  workers 

•  1,800  responses  that  appear  to  provide  unbiased  data  on  the 
entire  work  force 
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To  answer  the  four  research  questions,  we  used  both  qualitative  and  statisticai 
methodologies.  The  chart  above  describes  the  three  main  elements  of  the  research  effort. 

In  order  to  develop  an  understanding  of  the  Army  acquisition  system,  the  reforms,  their 
impact  on  various  acquisition  programs,  and  the  views  of  Army  personnel,  we  conducted 
nine  detailed  case  studies  of  Army  acquisition  programs.  The  nine  programs  were 
selected  to  represent  a  broad  spectrum  of  program  characteristics,  such  as  acquisition 
phase,  program  size  (ACAT  level),  type  of  technology  (military  or  commercial),  and 
program  location  (buying  command).  Each  case  study  was  based  on  extensive  document 
and  literature  review,  as  well  as  site  visits.  Site  visits  (2  to  3  days  in  length)  focused  on 
qualitative  data  collection  through  personnel  Interviews.  Interviews  were  conducted  with 
program  executive  officers  (PEOs),  program  managers,  divisional  directors,  and  line  staff 
at  all  levels  within  the  military,  engineering,  quality  assurance,  logistics,  contracting,  and 
safety  functions.  These  personnel  and  functions  were  selected  due  to  their  centrality  in  the 
acquisition  process  and  program  management.  In  addition,  complementary  inten/iews 
were  conducted  with  prime  contractor  personnel  involved  with  each  of  the  selected 
programs  in  order  to  get  contractor  perspectives  on  the  Army  reform  effort  and  its  effects. 
Data  from  these  case  studies  not  only  provided  context  for  our  research,  but  allowed  initial 
qualitative  analysis  and  the  development  of  hypotheses  on  the  reasons  for  support  and 
acceptance  of  reform.  Perhaps  most  importantly,  the  case  studies  provided  information 
essential  in  the  development  of  the  survey  discussed  below. 

Understanding  that  the  Army  was  attempting  major  changes  in  its  organization,  another 
element  of  our  research  agenda  included  discussions  with  six  nondefense  (mostly 
commercial)  organizations  believed  to  have  undertaken  successful  large-scale 
organizational  change  efforts.  These  interviews  helped  to  supplement  our  review  of  the 
literature  on  current  issues  in,  and  management  approaches  to,  organizational  change 
efforts.  Since  IPTs  and  supplier  partnerships  originated  in  the  commercial  sector,  these 
interviews  provided  essential  information  on  current  “best  practice”  in  the  field  with  potential 
applications  in  the  Army  context. 
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In  order  to  allow  quantitative  analysis  and  testing  of  hypotheses,  our  research  method 
included  a  comprehensive  survey  of  the  Army  acquisition  work  force.  As  the  chart  notes, 
we  surveyed  a  randomly  selected  sample  of  3,000  workers  (DAWIA  [Defense  Acquisition 
Workforce  Improvement  Act]  designated).  In  addition,  the  sample  was  stratified  to  allow 
statistically  meaningful  comparisons  among  subgroups.  The  survey  was  broad,  eliciting 
views  on  the  three  reform  efforts,  employee  support  and  attitudes  toward  reform, 
perceptions  of  support  mechanisms  present,  and  perceived  level  of  compliance  with  the 
reforms.  The  survey  response  was  good,  with  approximately  1,800  surveys  returned — 
representing  a  response  rate  of  60  percent.  Our  analysis  of  these  responses  reveals  no 
significant  patterns  in  the  response  rates  (across  occupational  or  program  characteristics). 
This  suggests  that  the  sample  provides  unbiased,  statistically  valid  estimates  of  pervasive 
attitudes,  perspectives,  and  behaviors  for  the  acquisition  work  force  at  large. 
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Empirical  Methodology 

•  Statistical  analysis  (ordered  logistic  regressions)  of  reported  beliefs, 
attitudes,  and  behaviors 

•  Explanation  of  response  distribution  by  examining  multiple  factors 

-  Personal  characteristics:  education,  tenure,  rank 

-  Job  characteristics:  specialty,  location 

-  Management  factors:  program,  matrix,  integrated  product 
teams  (IPTs),  collocation,  senior  and  primary  rater 

-  Other  human  resource  factors:  training,  compensation  policy 

•  Efforts  to  link  beliefs,  attitudes,  and  reported  behaviors 

-  Difficult  to  separate  correlation  and  causation 

-  Identification  of  targets  of  opportunity 
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As  noted  in  the  previous  chart,  the  data  provided  by  the  survey  allowed  quantitative 
analysis  of  the  link  between  employee  attitudes,  organizational  support  mechanisms,  and 
reform  behavior.  Many  of  the  upcoming  charts  in  this  briefing  present  the  results  of  these 
quantitative  analyses.  Therefore,  it  is  useful  at  this  time  to  briefly  describe  the  empirical 
methodology  used  to  link  attitudes,  beliefs,  and  reform  behavior. 

We  conducted  statistical  analyses — ordered  logistic  regressions — of  beliefs,  attitudes,  and 
behaviors  reported  in  the  Army  acquisition  survey.  In  each  case,  we  attempted  to  explain 
the  distribution  of  reported  attitudes  and  behaviors  by  examining  multiple  factors 
simultaneously.  These  factors  included  personal  characteristics,  job  characteristics, 
management  factors,  and  other  human  resource  factors. 

We  also  attempted  to  link  beliefs,  attitudes,  and  reported  behaviors  with  each  other.  Of 
course,  it  is  difficult  to  separate  correlation  from  causation.  To  the  extent  possible,  we 
employed  available  econometric  techniques  for  identifying  causal  relationships,  especially 
when  we  considered  some  of  the  key  policy  variables  at  the  Army’s  disposal — such  as  IPT 
and  Roadshow  training.  Although  one  must  exercise  caution  in  interpreting  some  of  our 
results,  we  feel  confident  that  we  have  identified  several  important  targets  of  opportunity  for 
improving  implementation  of  acquisition  reform. 
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Briefing  Overview 


Overall  progress  of  reform 

Progress  in  reforming  miispecs  and  standards 

Implementation  of  integrated  product  teams  (iPTs) 

Movement  toward  industry  partnerships 

Recommendations  for  improving  reform 
impiementation 
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The  chart  above  presents  the  outline  of  this  briefing.  Each  of  the  major  policy  initiatives  will 
be  examined  in  turn,  starting  with  an  initial  assessment  of  the  overall  progress  of  reform. 
This  assessment  examines  work  force  perceptions  of  the  reform  effort  “writ  large” — cutting 
across  all  three  major  policy  initiatives.  This  assessment  of  progress  establishes  the 
context  within  which  reform  is  being  attempted,  it  inciudes  an  examination  of  employee 
perceptions  of  the  change  resulting  from  reform;  their  attitudes  toward  and  support  for 
reform;  an  assessment  of  workplace  incentives;  and  their  views  on  the  ievel  of 
organizationai  support  mechanisms  within  the  Army  acquisition  system  (see  page  2). 

This  assessment  wili  be  foliowed  by  an  examination  of  progress  in  reforming  miispecs  and 
standards — ^the  centerpiece  of  the  current  Army  reform  effort.  As  indicated  earlier,  the 
success  of  miispec  and  standard  elimination  is  iikely  to  be  affected  by  greater  use  of  iPTs. 
Therefore,  progress  in  instituting  IPTs  is  examined  next.  In  addition,  IPTs  are  an  integral 
part  of  broader  efforts  to  move  toward  a  more  cooperative  reiationship  with  industry.  This 
third  initiative  will  be  examined  as  weli.  Finaiiy,  the  briefing  wili  conclude  with  a  summary 
of  major  conclusions  and  recommendations  for  improving  the  implementation  of  all  three 
initiatives  and  acquisition  reform  in  general. 
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Reforms  Are  Resulting  in 

Perceived  Changes 

Better 

No 

Change 

Worse 

“Interaction  with  the  contractor” 

29% 

66% 

5% 

“interaction  with  externai  oversight” 

23% 

73% 

4% 

“Amount  of  discretion  i  have” 

39% 

53% 

8% 

“My  understanding  of  priorities” 

50% 

33% 

17% 

“How  we  collaborate” 

43% 

48% 

9% 
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Based  on  responses  to  the  survey  items  outlined  in  the  above  chart,  it  appears  that  the 
work  force  does  perceive  reai  change  taking  piace  in  the  acquisition  system.  It  is  important 
to  point  out  that  the  statistics  reported  above  indicate  the  perceptions  of  the  acquisition 
work  force.  They  do  not  indicate  that  reai  changes  are  in  fact  taking  place,  only  that  Army 
employees  perceive  change  in  these  factors.  Perception  is  most  likeiy  based  on  reality  and 
experience,  and  may  be  very  important  in  motivating  change  and  reform  behavior. 

Implicitiy,  the  Army  reforms  are  designed  to  improve  interaction  with  the  contractor  and 
externai  oversight;  increase  employee  discretion;  improve  the  understanding  of  acquisition 
priorities;  and  improve  the  way  empioyees  coiiaborate.  In  all  these  cases,  at  least  20 
percent  of  the  employees  surveyed  feel  matters  have  improved  (gotten  better).  The 
greatest  improvement  is  perceived  to  have  taken  piace  in  the  communication  of  acquisition 
priorities  and  in  the  process  of  collaboration  (50  and  43  percent,  respectively). 

It  is  worth  noting  that  a  sizabie  proportion  of  the  respondents  beiieve  that  “no  change”  has 
occurred  in  the  five  items  described  above.  We  choose  to  emphasize  the  significant 
number  of  respondents  indicating  change  for  the  better.  Since  this  reform  process  is  only  a 
little  over  two  years  old,  the  number  of  respondents  seeing  improvement  seems  significant. 
In  addition,  in  most  cases  fewer  than  10  percent  of  the  employees  surveyed  feel  the  reform 
initiatives  have  made  these  factors  worse. 
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Support  for  Reform  Exists 

Survey  statement 

Agree 

No 

Opinion 

Disagree 

“There  are  compelling  reasons  to  reform  the  process” 

87% 

8% 

5% 

“People  working  on  my  program  agree  with  reform” 

61% 

22% 

17% 

“The  reforms  outline  sensible  behaviors” 

51% 

33% 

16% 

“My  managers  are  not  willing  to  give  up  traditional  ways” 

34% 

22% 

44% 

“The  various  reforms  fit  together  in  an  integrated  fashion” 

48% 

24% 

28% 

“When  the  current  leadership  changes,  so  will  the  reforms” 

53% 

27% 

20% 

Arroyo  center 

RAND 

In  the  literature  and  our  interviews  it  became  apparent  that  factors  such  as  the  perceived 
“need”  for  reform,  adequacy  of  reform  initiatives,  co-worker  support,  and  leadership 
commitment  were  associated  with  “support”  for  reform.  The  items  in  the  chart  above  were 
devised  to  measure  these  elements  of  support. 

The  perceived  need  for  reform  appears  to  be  widely  shared  within  the  work  force.  Of  those 
surveyed,  87  percent  agreed  that  “there  are  compelling  reasons  to  reform  the  [acquisition] 
process.”  The  literature  on  organizational  change  and  the  approach  employed  by  the 
private-sector  firms  interviewed  in  this  study  indicate  that  a  shared  perception  of  the  need 
to  change  can  be  vital  to  a  successful  change  effort. 

A  significant  number  of  respondents  appear  to  believe  that  the  Army’s  current  reform 
efforts  address  this  “compelling  need.”  For  example,  51  percent  of  those  surveyed  believe 
the  current  reforms  “outline  sensible  behaviors.”  In  addition,  48  percent  believe  the 
reforms  “fit  together  in  an  integrated  fashion.”  It  appears  that  a  significant  number  of 
employees  believe  the  reforms  are  adequate  at  some  level. 

Another  important  element  affecting  support  is  the  perceived  support  of  co-workers.  Of 
those  surveyed,  61  percent  indicated  that  their  co-workers  “agree  with  reform.”  Much  of 
the  literature  on  organizational  change  indicates  that  middle-level  managers  can  be  a 
barrier  to  effective  change.  In  the  Army  context,  views  on  middle-level  management 
appear  mixed.  Of  those  surveyed,  44  percent  disagreed  with  the  statement  “My  managers 
are  not  willing  to  give  up  traditional  ways,”  while  34  percent  indicated  they  are.  Finally,  the 
permanence  of  the  reforms  does  appear  to  be  in  question  by  the  work  force.  When  asked 
if  the  policies  would  change  “when  the  current  leadership  changes,”  53  percent  of  those 
surveyed  indicated  they  thought  this  outcome  was  likely. 

While  many  believe  reform  is  necessary,  a  significant  minority  expressed  some 
reservations.  For  example,  34  percent  indicate  resistance  from  their  managers,  28  percent 
believe  that  that  the  reforms  do  not  fit  together  in  an  integrated  fashion,  and  over  50 
percent  believe  that  the  reform  efforts  will  not  live  beyond  the  tenure  of  current  leadership. 
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What  “Incentives”  Are  Important? 


Avoiding  personal  risk 

Average 

37% 

Civilian 

41% 

Military 

27% 

Maximizing  Income 

39% 

48% 

10% 

Receiving  medals/awards 

32% 

35% 

20% 

Gaining  peer  recognition 

59% 

62% 

51% 

Supporting  the  soldier 

98% 

98% 

100% 

Following  orders 

59% 

58% 

62% 

Doing  job  correctly 

99% 

99% 

100% 

Doing  job  quickly 

75% 

74% 

77% 

Achieving  program  goals 

97% 

97% 

98% 

Arroyo  center 

RAND 

In  order  to  assess  the  motivations  affecting  support  for  the  reform  initiatives,  we  asked  a 
series  of  questions  about  the  relevance  of  a  variety  of  personal  factors.  The  above  tabulation 
presents  the  percentage  of  those  surveyed  who  indicated  that  the  stated  “incentive”  was 
“important”  in  their  workplace  decisions — such  as  whether  or  not  to  implement  reform.  This 
percentage  is  reported  for  the  sample  as  a  whole  and  for  the  civilian  and  military  subgroups. 
Although  we  explored  differences  between  other  subgroups,  none  of  these  differences  were 
large,  nor  were  they  predictive  of  levels  of  support  or  reported  behaviors. 

Two  related  patterns  emerge  from  the  data: 

•  Personal  incentives  are  valued  more  highly  by  civilian  employees,  and 

•  Programmatic  incentives  are  viewed  as  important  by  both  groups. 

Clearly,  civilian  employees  value  personal  rewards  more  than  military  personnel  do.  The 
first  four  “incentives”  in  the  table  reflect  personal  rewards,  and  the  percentage  of  civilian 
employees  who  feel  these  are  “important”  exceeds  that  for  military  personnel  in  all  four 
cases.  For  example,  37  percent  of  the  sample  indicated  that  avoiding  personal  risk  is 
important.  Civilian  personnel  were  more  likely  to  emphasize  this  factor  (41  percent) 
compared  with  the  military  (27  percent).  Almost  half  of  all  civilians  cite  maximizing  income  as 
being  important,  while  only  10  percent  of  the  military  personnel  state  that  income  is  important. 

The  final  five  “incentives”  can  be  considered  programmatic — they  pertain  to  organizational 
and/or  acquisition  program  outcomes,  such  as  “achieving  program  goals”  and  “supporting  the 
soldier.”  These  programmatic  goals  were  cited  as  being  important  by  virtually  all  the  survey 
respondents,  both  military  and  civilian. 

Despite  these  differences,  our  analysis  indicated  that  variations  in  the  value  placed  on 
general  incentives  (such  as  the  above)  do  not  appear  to  be  highly  predictive  of  overall 
support  for  reform  or  reform  behavior.  Greater  predictive  utility  is  derived  from  employee 
perceptions  of  specific  personal  and  programmatic  outcomes.  This  distinction  will  be 
explained  and  these  results  explored  in  upcoming  sections. 
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Support  Mechanisms  Need  to  Be  Aligned 

Survey  statement 

Agree 

No 

Opinion 

Disagree 

“My  pay  is  determined  by  the  success  of  the  team” 

13% 

24% 

63% 

“My  managers  empower  me  to  make  decisions” 

66% 

12% 

22% 

“There  is  good  data  on  past  performance” 

27% 

28% 

45% 

“Information  systems  meet  our  new  needs” 

32% 

22% 

46% 

“Reform  communication  is  clear  and  consistent” 

30% 

24% 

46% 

“I’m  not  sure  what  is  expected  of  me” 

26% 

28% 

46% 

“1  have  received  sufficient  training  to  work  w/  reforms” 

37% 

18% 

45% 

“Even  with  all  the  changes,  1  feel  my  job  is  safe” 

30% 

19% 

51% 

Arroyo  center 

RAND 

Although  the  work  force  appears  to  support  reform,  it  is  clear  that  support  is  not  unanimous 
and  that  efforts  should  be  considered  to  increase  support  levels.  In  particular,  support 
mechanisms  can  be  better  aligned.  Several  items  in  the  survey  were  designed  to  measure 
the  perceived  presence  of  organizational  support  mechanisms  in  the  Army  acquisition 
system.  These  are  listed  in  the  table  above. 

While  this  list  is  not  inclusive  of  all  potential  support  mechanisms  or  factors,  it  does  identify 
beliefs  related  to  several  key  mechanisms,  such  as  human  resource  policies,  empowerment, 
measurement  and  information  systems,  communication,  and  training.  Each  of  these  factors 
is  identified  as  important  in  facilitating  large-scale  organizational  change  in  the  literature  and 
through  our  interviews  with  successful  nongovernmental  organizations. 

Current  reform  efforts  appear  to  have  made  the  most  progress  in  empowering  acquisition 
employees.  Of  those  surveyed,  66  percent  feel  that  their  managers  empower  them  to  make 
decisions. 

The  perceived  presence  of  other  support  mechanisms  is,  however,  much  lower.  On  the 
human  resources  side,  few  employees  feel  their  pay  is  linked  to  team  success  (13  percent). 
Measurement  and  information  systems  are  considered  inadequate  with  45  percent  of 
respondents  disagreeing  with  the  statement  that  past  performance  data  are  “good”  and  46 
percent  disagreeing  with  the  statement  that  current  information  systems  “meet  our  needs.” 
Views  on  the  adequacy  of  communication  related  to  the  reforms  are  mixed.  On  the  negative 
side,  46  percent  of  respondents  disagree  with  the  statement  “Reform  communication  is  clear 
and  consistent.”  On  the  other  hand,  46  percent  disagree  with  the  statement  “I’m  not  sure 
what  is  expected  of  me.”  Finally,  a  significant  number  of  respondents — 45  percent— appear 
to  feel  they  have  not  received  sufficient  training  to  work  with  the  reforms. 

Thus,  while  support  for  reform  exists,  it  should  be  possible  to  increase  support  levels  through 
a  greater  understanding  of  why  employees  might  resist  reform  and  an  emphasis  on  changing 
attitudes  and  behavior  through  the  alignment  of  organizational  support  mechanisms.  The 
next  three  sections  will  examine  each  of  the  major  initiatives  and  provide  general 
recommendations  on  how  the  Army  can  approach  improving  the  implementation  of  its 
acquisition  reform  initiatives. 
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Briefing  Overview 


•  Overall  progress  of  reform 

•  Progress  in  reforming  milspecs  and  standards 

•  Implementation  of  integrated  product  teams  (IPTs) 

•  Movement  toward  industry  partnerships 

•  Recommendations  for  improving  reform 
implementation 


Arroyo  center 


RAND 


We  first  examine  progress  in  reforming  milspecs  and  standards. 

As  indicated  earlier,  the  Army  has  adopted  a  “zero  tolerance”  approach  toward  the  use  of 
milspecs  and  standards  in  the  acquisition  process.  Specifically,  milspecs  and  standards 
are  not  to  be  included  in  requests  for  proposal  (RFPs)  and  statements  of  work  (SOWs) 
related  to  Army  acquisition  programs.  If  a  program  manager  desires  to  include  milspecs 
and  standards,  a  waiver  must  be  granted  by  the  relevant  acquisition  authority  (most  often 
SARDA  or  the  PEO). 

The  next  nine  charts  will  present  our  analysis  of  the  current  state  of  milspec  and  standard 
reform  within  the  Army  acquisition  system.  We  will  begin  by  assessing  employee  beliefs 
about  the  likely  effects  of  reform  and  their  overall  support  for  the  effort.  Having  established 
this  context,  we  will  then  identify  how  support  levels  vary  within  the  work  force  and  whether 
the  level  of  support  appears  linked  to  actual  elimination  of  milspecs  and  standards 
(behavior).  Having  identified  the  level  of  support  and  its  effect  on  behavior,  we  will 
examine  how  beliefs  about  the  effects  of  milspec  and  standard  elimination  affect  support 
levels.  This  process  should  allow  the  identification  of  important  perceptions  and 
misconceptions  affecting  the  willingness  of  employees  to  implement  milspec  and  standard 
reform.  Having  identified  important  beliefs,  we  will  next  examine  the  effect  of  training  and 
communication  programs  in  changing  beliefs.  Finally,  we  will  summarize  the  major 
conclusions  and  provide  some  general  policy  recommendations  on  how  to  improve  the 
implementation  of  milspec  and  standard  reform. 
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Overall  Views  of  Mllspec  Reform  Are 
Generally  Positive 

65%  of  respondents  support  the  elimination  of  milspecs  and 
standards 

72%  believe  that  reform  wili  reduce  current  program  costs 

88%  believe  that  eliminating  milspecs  will  encourage  the 
entry  of  commercial  firms 

Beliefs  on  a  range  of  other  effects  are  mixed  (about  50-50) 

•  Quaiity 

•  Life-cycle  costs 

•  User  satisfaction 


Arroyo  center 


As  this  chart  shows,  attitudes  toward  milspec  reform  are  generally  positive,  with  65  percent 
of  the  respondents  indicating  that  they  support  the  policy. 

A  basic  approach  in  our  research  is  to  examine  the  relationship  between  beliefs  about 
reform  and  support  levels.  The  next  two  items  on  the  chart  show  that  beliefs  about  two 
“key”  outcomes  desired  of  reform — reducing  current  program  costs  and  encouraging  the 
entry  of  new  firms— are  also  positive.  Specifically,  72  percent  of  respondents  believe  that 
milspec  reform  will  reduce  costs  and  88  percent  believe  that  it  will  encourage  the  entry  of 
commercial  firms. 

While  beliefs  about  some  key  outcomes/effects  are  very  positive  in  the  work  force,  beliefs 
about  the  effect  of  reform  on  other  outcomes  are  more  mixed.  The  final  item  in  the  chart 
indicates  that  on  outcomes  such  as  product  quality,  life-cycle  costs,  and  user  satisfaction, 
only  about  50  percent  of  the  surveyed  work  force  feels  that  milspec  and  standard  reform 
will  result  in  desirable  outcomes  (such  as  increasing  product  quality,  lowering  life-cycle 
costs,  and  increasing  user  satisfaction).  Outcomes  such  as  these,  where  variability  in 
beliefs  is  greatest,  may  help  us  in  understanding  why  35  percent  of  those  surveyed  do  not 
support  milspec  and  standard  reform. 
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Should  Milspecs  and  Standards  Be  Eliminated? 


Survey  statement:  “I  support  the  elimination  of  milspecs  and  standards” 

Strongly  Agree  Disagree  Strongly 

agree  Agree  somewhat  somewhat  Disagree  disagree 


;  Engineer 

12% 

26% 

27% 

15% 

11% 

9% 

QA 

5% 

15% 

23% 

18% 

19% 

20% 

Contract 

26% 

35% 

22% 

9% 

4% 

4% 

Military 

24% 

34% 

23% 

9% 

6% 

4% 

Matrix 

10% 

23% 

27% 

16% 

13% 

11% 

Tenure  >  20 

4% 

14% 

23% 

18% 

20% 

21% 

Projections  are  based  on  logistic  model  estimates 

Organization,  phase,  and  program  size  are  not  significant  predictors 


Ajroyo  center  rand 


This  chart  predicts  survey  responses  to  the  statement  “I  support  the  elimination  of  milspecs 
and  standards.”  The  projections  are  based  on  parameter  estimates  from  an  ordered 
logistic  model  that  linked  the  distribution  of  responses  (from  “strongly  agree”  through 
“strongly  disagree”)  to  multiple  characteristics.  In  other  words,  the  figures  reported  above 
are  projections  of  how  responses  to  the  statement  change  based  on  important 
characteristics  of  the  population;  some  of  these  characteristics  are  listed  in  the  leftmost 
column  of  the  table  above. 

The  first  line  of  the  chart  (shaded)  is  provided  as  a  baseline  for  comparison.  This  line  is  the 
predicted  response  of  a  typical  engineer.  Engineers  were  chosen  as  the  baseline  since 
they  have  a  distribution  of  responses  that  is  generally  reflective  of  the  average  member  of 
the  acquisition  work  force.  As  indicated,  approximately  65  percent  of  engineers  responded 
that  they  “strongly  agree,”  “agree,”  or  “somewhat  agree”  with  the  statement  of  support  for 
milspec  and  standard  reform. 

The  next  five  lines  of  the  table  project  the  response  pattern  if  this  typical  engineer’s 
characteristics  are  changed  in  some  manner.  These  projections  are  based  on  the 
parameter  estimates  of  the  following  ordered  logistic  model  (in  general  form): 

(Response  Distribution)  =  /  (career  field,  job  characteristics,  personal  characteristics, 

organization  characteristics,  and  program  characteristics) 

Of  the  many  variables  measuring  career  field  and  job,  personal,  organizational,  and 
program  characteristics,  the  six  identified  in  the  chart  above  were  the  most  significant  in 
predicting  the  response  distribution.  When  the  typical  engineer  is  transformed  into  a 
quality  assurance  employee  holding  all  other  characteristics  constant — ^as  reported  in  line 
two  of  the  table — we  can  see  that  agreement  with  the  survey  statement  plummets  from  65 
percent  to  43  percent — a  statistically  significant  reduction.  By  varying  the  career  field  and 
holding  all  other  variables  constant,  this  result  provides  the  marginal  effect  of  the  QA 
career  field  on  projected  responses  (compared  with  engineers). 
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In  fact,  career  field  appears  to  be  the  most  important  characteristic  expiaining  responses  to 
the  support  statement.  While  the  effect  of  being  a  QA  employee  lowers  support,  the  chart 
shows  that  the  effect  of  being  in  the  contracting  or  military  career  fields  (other  factors  held 
constant)  increases  the  overall  support  level  from  65  percent  for  the  typical  engineer  to  83 
percent  for  contracting  and  81  percent  for  military  personnel.  Not  only  are  contracting  and 
military  personnel  more  supportive  than  the  typical  engineer,  they  are  almost  twice  as 
supportive  as  QA  personnel. 

The  fifth  line  describes  the  isolated  (and  independent)  effect  of  being  a  matrix  employee 
(not  affiliated  with  a  program  management  office).  In  this  case  the  response  of  a  typical 
engineer  will  fall  from  65  percent  agreement  to  60  percent.  While  statistically  significant 
(at  the  95  percent  confidence  level),  the  effect  of  matrix  affiliation  does  not  appear  to  be 
large  in  a  practical  sense.  For  example,  while  12  percent  of  engineers  strongly  agree  with 
the  statement  of  support,  this  percentage  falls  to  10  percent  for  an  engineer  who  is 
assigned  to  a  functional  buying  command  (i.e.,  the  matrix).  This  matrix  effect  would  be 
the  same  for  all  the  career  fields,  including  engineers,  QAs,  logisticians,  business,  and 
contracting,  as  well  as  military  personnel. 

It  is  worth  noting  that  much  of  the  observed  difference  between  matrix  and  program 
management  personnel  can  be  attributed  to  systematic  variations  between  the  mix  of 
characteristics  that  generally  prevail  in  each  of  these  situations.  For  example,  matrix 
personnel  are  more  likely  to  be  in  quality  assurance  and  less  likely  to  be  military 
employees.  These  characteristics — career  field— and  not  the  matrix  affiliation  are  more 
likely  the  sources  for  the  observed  variation  in  support  between  individuals  in  the  Army 
acquisition  corps. 

The  final  line  indicates  the  marginal  effect  of  tenure  (measured  as  having  over  20  years 
experience  in  acquisition).  As  reported,  support  for  milspec  and  standard  reform  falls 
dramatically  for  those  (in  all  occupations)  with  long  tenure  in  acquisition.  Based  on  our 
interviews  with  acquisition  personnel,  this  lower  support  level  could  be  the  result  of  caution 
on  the  part  of  senior  employees  who  fear  for  their  job  and  position. 

Finally,  several  other  variables  were  analyzed,  but  none  had  significant  effects  on  the 
response  distribution.  Whiie  our  initial  interviews  with  acquisition  personnel  suggested  that 
factors  such  as  organization  (specific  buying  command  or  PEO),  program  phase  (e.g., 
development  versus  manufacturing),  and  program  size  (Acquisition  Category  [ACAT]  level) 
would  have  a  large  effect  on  employee  support  and  attitudes,  this  does  not  appear  to  be 
the  case.  None  of  these  factors  were  statisticaliy  significant  in  predicting  the  response 
distribution  of  support  for  milspec  and  standard  reform.  Although  simple  cross  tabulations 
wouid  indicate  differences  along  these  characteristics,  most  of  these  can  be  attributed  to 
systematic  variations  in  the  personnei  mix  (such  as  career  field  and  tenure). 
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Reported  Behaviors  Vary  Based  on 
General  Support  for  Milspec  Reform 

Agreement 

Survey  statement 

Those  supporting 
milspec  reform 

Those  not 
supporting 

“1  do  not  use  milspecs  or  standards” 

74% 

52% 

“1  rewrite  milspecs  into  performance  specs” 

60% 

53% 

“1  communicate  a  desire  for  miispecs  to 
contractor” 

5% 

17% 

“Request  a  waiver” 

9% 

33% 

“i  use  milspecs  and  standards  until  directed 
otherwise” 

14% 

26% 

“Rely  on  contractor  to  propose  non-milspec 
approach” 

60% 

60% 

Arroyo  center 

RAND 

Is  there  a  link  between  support  and  reform  behavior?  A  simple  analysis  of  reform 
behaviors  for  those  who  support  reform  and  those  who  do  not  suggests  there  is  such  a  link. 

To  illustrate  this  point,  the  sample  was  divided  into  two  groups  depending  on  the  degree  of 
support  for  milspec  and  standard  reform.  In  the  survey,  each  group  was  asked  the  extent 
to  which  they  agreed  or  disagreed  that  they  perform  a  variety  of  behaviors  consistent  with 
either  implementing  or  circumventing  milspec  and  standard  reform.  For  example,  the  first 
statement,  “I  do  not  use  milspecs  or  standards,”  is  the  most  direct  measurement  of 
implementing  milspec  and  standard  reform.  Other  statements,  such  as  “I  communicate  a 
desire  for  milspecs  to  the  contractor,”  can  be  considered  a  description  of  a  behavior  that 
circumvents  the  intention  of  milspec  and  standard  reform. 

Based  on  a  simple  comparison  of  the  percent  of  each  subgroup  who  agree  with  the 
indicated  behavioral  statement,  it  appears  that  support  is  linked  to  behavior.  For  example, 
74  percent  of  those  professing  support  for  milspec  reform  agree  with  the  first  (most  direct) 
behavioral  statement.  In  contrast,  52  percent  of  those  claiming  nonsupport  agree  that  they 
no  longer  use  milspecs  and  standards. 

Sixty  percent  of  the  supporters  report  that  they  rewrite  milspecs  into  performance  specs,  in 
comparison  with  53  percent  of  those  not  reporting  support. 

A  small  percentage  of  individuals  admit  that  they  communicate  a  desire  for  milspec 
approaches  to  contractors,  although  the  percentage  is  over  three  times  higher  for 
nonsupports  (17  versus  5  percent).  In  addition,  almost  twice  as  many  of  the  nonsupporters 
. .  use  milspecs  and  standards  until  directed  othenwise”  (a  delaying  tactic)  when 
compared  to  those  who  support  (26  versus  14  percent,  respectively). 

It  is  interesting  to  note  that  significant  numbers  (33  percent)  of  those  resistant  to  milspec 
reform  request  waivers  (compared  with  9  percent  of  those  supporting  the  reform  initiative). 
This  was  slightly  unexpected,  since  earlier  interview  data  suggested  that  even  those  who 
did  not  support  the  initiative  were  reluctant  to  request  a  waiver.  This  reluctance  reportedly 
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stemmed  from  a  fear  that  requesting  a  waiver  would  be  frowned  upon  by  upper 
management.  This  fear  does  not  appear  to  be  as  pronounced  as  we  originally  assumed. 

It  is  also  interesting  to  note  that  an  equal  percentage  in  both  subgroups  report  that  they 
“rely  on  the  contractor  to  propose  non-milspec  approach.”  This  is  a  behavior  consistent 
with  the  Army’s  policy. 
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Beliefs  Affecting  Support  for  Milspec  Reform 

A  B 


Eliminating 
milspecs  and 
standards  will... 

Increase  product 
quality 

Make  more  work 

Increase  life-cycle 
costs 

Reduce  my  authority 
Require  new  skills 
Reduce  program  costs 

Arroyo  center 


Effect  on 
support 

+  +  + 


+ 
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Who  believes  in  these  outcomes? 


QA  Log  Cont 
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Mil 


Eng 


iZZZj 

Less  than  Same  as  More  than 
Engineer  Engineer  Engineer 


RAND 


This  chart  presents  the  results  of  two  separate  statistical  analyses.  These  analyses 
answer  the  following  two  questions: 

•  (A)  Which  outcome  beliefs  appear  to  have  an  effect  on  an  employee’s  support 

for  milspec  and  standard  reform? 

•  (B)  Which  career  groups  hold  these  beliefs? 

(A)  In  the  Army  acquisition  survey,  respondents  were  asked  a  series  of  questions  about  the 
“outcomes”  they  expected  to  result  from  an  elimination  of  milspecs  and  standards.  These 
outcome  beliefs  were  used  as  independent  variables  in  a  regression  to  predict  responses 
to  the  statement:  “I  support  the  elimination  of  milspecs  and  standards.”''  The  first  column 
lists  the  six  outcome  beliefs  that  were  found  to  be  statistically  significant  in  predicting  the 
extent  to  which  an  individual  supported  milspec  and  standard  reform.  A  plus  mark  (+) 
indicates  a  positive  correlation  (the  greater  the  belief  in  a  given  outcome,  the  greater  the 
support)  and  a  minus  sign  (-)  reflects  a  negative  relationship  (the  greater  the  belief  in  a 
given  outcome,  the  lower  the  support).  A  double  or  triple  plus  or  minus  sign  (e.g.,  +++, 
or  -  -)  signals  the  magnitude  of  the  effect. 

As  the  +  and  -  signs  in  column  A  indicate,  beliefs  about  the  effect  of  reform  on  product 
quality  were  the  most  important  in  predicting  support  for  milspec  and  standard  reform.  The 
three  plus  signs  (+++)  indicate  that  individuals  who  believed  that  reform  would  “increase 
product  quality”  were  much  more  likely  to  support  the  elimination  of  milspecs  and 
standards.  Likewise,  respondents  who  believed  that  reform  would  “require  new  skills”  (+) 
and  “reduce  program  costs”  (+)  were  also  more  likely  to  support  reform  (although  the  effect 
of  these  beliefs  was  not  as  large).  On  the  other  hand,  individuals  who  believed  that  reform 
would:  “make  more  work”  (-  -),  “increase  life-cycle  costs”  (-),  and  “reduce  my  authority”  (-) 
were  less  likely  to  support  reform. 


■■The  basic  model  used  in  part  A  is  (Response  Distribution)  =  f  (outcome  beiiefs). 
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These  beliefs  obviously  relate  to  potentially  negative  outcomes  of  reform,  so  respondents 
who  believed  negative  outcomes  were  likely  (or  positive  outcomes  less  likely)  are  less 
supportive.  Conversely,  those  who  believed  negative  outcomes  were  unlikely  (or  positive 
outcomes  more  likely)  are  more  supportive. 

(B)  Having  identified  which  beliefs  are  most  important  in  predicting  support,  it  will  be  useful 
to  understand  which  elements  of  the  work  force  hold  these  beliefs  or,  perhaps  as 
important,  hold  counter  beliefs.  The  results  of  this  analysis  are  provided  in  the  section 
marked  (B)  on  this  chart. 

Section  (B)  compares  the  extent  to  which  an  outcome  belief  is  likely  to  be  held  by  each 
career  group.  Once  again,  engineers  serve  as  our  base  for  comparison.  Each  row  under 
section  (B)  represents  a  separate  analysis  for  each  outcome  belief.^  For  example, 
compared  with  engineers,  logisticians  and  contracting  personnel  are  more  likely  to  believe 
that  “eliminating  milspecs  and  standards  will  increase  product  quality”  (row  one).  Not 
surprisingly,  quality  assurance  personnel  are  more  likely  not  to  believe  that  quality  will  rise 
under  milspec  and  standard  reform.  This  is  likely  the  most  important  factor  in  explaining 
the  negative  support  levels  of  QA  personnel  (see  earlier  charts). 

The  second  most  important  belief  in  predicting  support  was  the  belief  that  reform  would 
“make  more  work."  Employees  who  believed  this  were  much  less  likely  to  support  reform. 
As  section  (B)  shows,  all  five  career  groups  essentially  share  this  outcome  belief— they  all 
hold  the  belief  at  the  same  level  as  an  engineer. 

Those  who  believe  life-cycle  costs  will  rise  as  a  result  of  reform  are  less  supportive. 
Contracting  and  military  personnel  are  less  likely  to  believe  that  this  negative  outcome  will 
result.  Engineers,  logisticians,  and  QA  personnel  all  seem  to  share  the  same  level  of  belief 
with  regard  to  life-cycle  costs. 

Those  who  believe  that  reform  will  reduce  their  authority  are  also  more  likely  to  oppose 
reform.  Not  surprisingly,  QA  personnel  are  more  likely  to  believe  that  reform  will  reduce 
their  authority  (when  compared  to  an  engineer),  while  contracting  personnel  are  relatively 
more  likely  to  believe  their  authority  will  increase  under  reform. 

Those  who  believe  that  reform  will  require  them  to  learn  new  skills  are  more  likely  to 
support  reform.  As  it  turns  out,  logisticians  are  much  more  likely  to  believe  that  they  will 
need  new  skills,  thus  contributing  to  their  positive  outlook  on  reform. 

Finally,  those  who  believe  program  costs  will  fall  with  the  elimination  of  milspecs  and 
standards  are  more  likely  to  support  the  reform.  Only  one  career  group — QA — questions 
this  potential  positive  outcome.  In  other  words,  QA  personnel  are  less  likely  to  believe  that 
program  costs  will  fall  as  a  result  of  reform.  This  finding  further  explains  the  negative  view 
of  this  career  group. 

The  large  effect  of  beliefs  related  to  product  quality,  coupled  with  the  significant  effect  of 
beliefs  about  life-cycle  and  program  costs,  suggests  that  beliefs  about  program  outcomes 
are  somewhat  more  important  in  predicting  support  than  are  beliefs  about  personal 
outcomes.  It  is  important  to  note  that  personal  outcomes,  such  as  making  more  work, 
reducing  authority,  and  requiring  new  skills,  are  important  in  predicting  support.  Relatively 
speaking,  however,  it  appears  that  programmatic  concerns  predominate  in  determining 
support.  This  observation  will  be  important  in  the  design  of  policies  meant  to  promote  or 
facilitate  the  successful  implementation  of  milspec  and  standard  reform. 


^The  analysis  conducted  here  used  an  ordered  logistic  regression  to  predict  the  distribution  of 
responses  (from  “strongly  agree”  to  “strongly  disagree”)  to  each  of  the  items  noted  in  the  chart  based 
on  career  group.  Therefore,  the  basic  form  of  the  models  is 

(Response  Distribution)  =/  (career  field). 
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Training  Strengthens  Reported  Beiiefs 


Survey  statement  Benchmark 

Roadshow  IV 

IPT 

training 

“I  support  the  elimination  of 
milspecs  and  standards” 

(agree  and  strongly  agree) 

38% 

46% 

45% 

“Reform  goals  are  communicated 
in  a  clear  and  concise  fashion” 

(agree  and  strongly  agree) 

62% 

73% 

74% 

“Eliminating  milspecs  and  standards 
wilt  reduce  current  program  costs” 
(quite  likely  and  very  likely) 

35% 

40% 

45% 

“Eliminating  milspecs  and  standards 
will  increase  product  quality” 

(quite  likely  and  very  likely) 

23% 

24% 

31% 

Arroyo  center 

RAND 

We  also  assessed  the  extent  to  which  participation  in  IPT  training  and  Roadshow  IV — the 
two  major  forms  of  training  meant  to  facilitate  acquisition  reform  generally  and  milspec  and 
standard  reform  specifically — affected  support  and  related  beliefs.  To  do  this,  we  included 
a  “dummy”  variable  (equal  to  one  with  attendance,  zero  othenwise)  in  our  regression 
analyses  indicating  whether  or  not  an  individual  had  participated  in  Roadshow  IV  or  IPT 
training. 

The  first  column  of  percentages  reports  a  benchmark  response  rate  (essentially  the  sample 
average)  to  the  listed  survey  statements.  Here  the  percentages  indicate  the  share  of 
respondents  who  indicated  that  they  “strongly  agree”  or  “agree”  with  each  of  the  four 
statements  in  the  chart.  For  example,  38  percent  of  the  sample  indicated  that  they  agree 
(or  strongly  agree)  with  the  first  statement:  “I  support  the  elimination  of  milspecs  and 
standards.”  Based  on  regression  parameters  (holding  ail  other  characteristics  constant), 
an  otherwise  identical  individual  attending  Roadshow  IV  would  report  agreement  46 
percent  of  the  time.  The  effect  of  IPT  training  was  similar,  with  45  percent  reporting 
support  for  milspec  and  standard  reform.  Thus,  the  isolated  effect  for  each  type  of 
training — Roadshow  IV  and  IPT  training — ^was  a  20  percent  increase  in  the  number  of 
respondents  indicating  that  they  “strongly  agree”  or  “agree”  with  the  statement  of  support 
for  milspec  and  standard  reform. i 

The  impact  of  training  on  the  three  remaining  beliefs  was  similar.  Both  Roadshow 
attendance  and  IPT  training  served  to  enhance  understanding  of  reform  objectives.  Those 
attending  both  types  of  training  are  also  more  likely  to  believe  that  eliminating  milspecs  and 
standards  will  reduce  current  program  costs.  In  this  case,  IPT  training  has  a  larger  positive 
effect  on  beliefs. 


''Of  course,  this  correlation  may  be  spurious,  with  individual  support  as  well  as  participation  in  training 
being  due  to  some  other,  unmeasured,  phenomenon.  To  test  this,  we  employed  standard 
econometric  techniques  for  assessing  the  potential  problem.  We  were  able  to  reject  the  hypotheses 
that  Roadshow  attendance  or  IPT  training  were  endogenous  variables.  Thus,  our  evidence  strongly 
suggests  that  there  is  a  high  payoff  to  modest  training  investments. 
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It  is  interesting  to  note,  however,  that  Roadshow  IV  attendance  did  little  to  change  beliefs 
related  to  reform  effects  on  product  quality  (the  key  outcome  belief  identified  on  the 
previous  chart).  IPT  training,  on  the  other  hand,  improved  perceptions  of  product  quality 
resulting  from  elimination  of  milspecs  and  standards.  This  finding  suggests  that  the 
Roadshow  training  program  did  a  poor  job  in  convincing  participants  of  the  positive  quality 
outcomes  to  be  expected  as  a  result  of  eliminating  milspecs  and  standards. 


Who  Has  Received  Training? 


Roadshow 

IV 

IPT 

Training 

Engineering 

22% 

44% 

Quaiity  assurance 

22% 

52% 

Logistics 

19% 

51% 

Contracting 

29% 

36% 

Miiitary 

22% 

59% 

Sample  average 

23% 

52% 

>Arroyo  center  rand 


Having  identified  the  potential  effects  of  training  on  support  levels  and  key  outcome  beliefs, 
it  is  useful  to  examine  the  extent  to  which  the  acquisition  work  force  has  participated  in  the 
two  training  programs. 

As  indicated  in  this  table,  only  23  percent  of  the  surveyed  work  force  attended  Roadshow 
IV.  This  suggests  that  there  is  significant  potential  in  expanding  the  number  of  individuals 
who  are  exposed  to  training.  Also  note  that  all  occupational  fields  appear  to  have  attended 
Roadshow  IV  with  similar  frequency.  For  example,  the  percent  attending  from  the  QA 
ranks  is  the  same  as  for  military  personnel  (22  percent).  This  illustrates  the  point  that 
training  sessions  are  not  necessarily  drawing  individuals  who  are  predisposed  to 
supporting  reform. 

In  addition,  52  percent  of  all  respondents  have  undergone  IPT  training  (range  by  career 
group  from  36  to  59  percent).  This  suggests  that  some  cost-effective  strategies — such  as 
expanding  training — exist  for  influencing  prevailing  attitudes  toward  reform  in  a  positive 
direction. 
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Conclusions:  Milspec  and  Standards  Reform 

•  Resistance  is  related  to  functional  domains 

-  Resistance  is  magnified  with  long  tenure  in  acquisition 

•  Programmatic  incentives  and  outcome  beliefs  play  largest  role 

-  Concerns  center  on  quality,  life-cycle  costs,  program  costs 

•  Personal  Incentives  and  outcome  beliefs  play  smaller  role 

-  Job  security,  risk  avoidance  are  important  for  civilians 

-  Workload  concerns  emerge 

•  Structural  issues  are  less  important  than  previously  believed 

-  Matrix  is  a  smaller  factor  than  expected 

-  PEO  and  command  affiliation  appear  insignificant 

-  Yet,  functional  hierarchy  is  most  resistant 

•  Training  can  change  beliefs  and  reported  behaviors 

•  Experience  with  an  effective  IPT  enhances  support 
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We  found  that  variation  in  support  and  behavior  is  highly  correlated  with  functional  domains 
or  career  groups.  Program  management  and  contracting  personnel  are  clearly  the  most 
supportive  of  milspec  and  standard  reform.  Quality  assurance  personnel,  on  the  other 
hand,  are  the  most  opposed  to  the  reform.  Besides  functional  domains,  resistance  also 
appears  to  be  centered  among  employees  with  over  20  years  of  experience  in  the 
acquisition  process. 

Programmatic  incentives  and  outcome  beliefs  play  a  large  role  in  explaining  these 
systematic  variations  in  support  for  reform.  Programmatic  concerns — ^which  can  also  be 
functional  concerns  (i.e.,  quality  and  QA  personnel) — such  as  quality,  life-cycle  costs,  and 
program  costs  appear  to  drive  support  for  reform  among  Army  employees. 

While  less  important  than  programmatic  concerns,  beliefs  about  personal  outcomes  are 
also  important.  Although  job  security  and  risk  avoidance  are  important  to  civilian 
employees,  they  are  not  dominant  factors.  There  Is,  however,  widespread  concern  over 
increases  in  the  workload  and  reductions  in  personal  authority. 

It  is  interesting  to  note  that  structural  or  organizational  issues  are  less  important  than  many 
of  the  employees  and  managers  we  interviewed  believe.  The  matrix  organization  has  a 
small  effect,  once  occupational  affiliation  Is  taken  into  account.  The  PEO  and  command 
affiliations  are  insignificant.  However,  the  upper  levels  of  the  functional  hierarchy  remains 
the  most  resistant  to  reform,  perhaps  because  of  a  vested  interest  in  the  organizational 
status  quo. 

The  evidence  strongly  suggests  that  modest  amounts  of  training  can  change  beliefs  and 
reported  behaviors.  In  addition,  although  not  reported  earlier,  our  analysis  indicates  that 
experience  with  an  effective  IPT  enhances  support  for  milspec  and  standard  reform.  As 
indicated  earlier,  IPTs  can  play  an  important  role  in  facilitating  the  implementation  of 
milspec  and  standard  reform — we  will  have  more  to  say  on  this  topic  in  the  next  section  of 
the  briefing. 


■“An  effective  IPT  is  one  that  has  a  charter. 
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Recommendations:  Improving 
Implementation  of  Milspec  Reform 

•  Target  training/communications  efforts  to  those  who  are  resistant 

-  Don’t  “preach  to  the  choir”;  military  and  contracting  personnel  are 
already  largely  supportive 

-  Focus  on  resistant  elements  of  QA,  engineering,  and  logistics 
functions 

-  Decentralize  and  expand  training  efforts  to  reach  more  of  target 
work  force 

-  Emphasize  technical  concerns — e.g.,  quality  and  cost  impacts 

-  Improve  the  Roadshow,  emphasize  IPT  training 

•  Some  human  resource  policies  should  be  considered 

-  Minimize  impact  on  workload  by  reallocating  resources 

-  Consider  retraining  and  finding  new  roles  for  senior  employees 

•  Break  down  functional  resistance  through  effective  IPTs 

Arroyo  center  rand 


The  conclusions  on  the  previous  chart  suggest  several  policy  options  for  improving  the 
implementation  of  milspec  reform.  First,  it  would  be  useful  to  target  additional  training  and 
communications  to  those  who  are  most  resistant,  especially  elements  of  the  QA,  logistics, 
and  engineering  functions.  These  efforts  should  address  technical  concerns,  such  as 
quality  and  life-cycle  cost  impacts.  This  can  best  be  accomplished  through  decentralized 
training  efforts  that  are  tailored  to  address  programmatic  issues.  Despite  impressive 
effects  on  the  attitudes  of  attendees,  it  is  clear  that  the  Roadshow  approach  can  be 
improved  on  this  score.  To  the  extent  possible,  continued  IPT  training  for  acquisition 
employees  is  to  be  encouraged. 

Changes  in  human  resource  management  policies  may  also  have  some  potential  to 
improve  implementation  of  milspec  and  standard  reform.  First,  it  would  make  sense  to 
minimize  workload  concerns  by  reallocating  resources — at  least  to  explicitly  recognize  that 
reform  may  require  an  initial  upfront  investment  in  people  and  training.  Also,  given  the 
resistance  among  senior  employees  and  within  some  career  groups,  it  might  make  sense 
to  consider  retraining  and/or  relocation  of  some  employees. 

Finally,  functional  resistance  has  been  identified  as  a  key  barrier  to  the  implementation  of 
milspec  and  standard  reform.  Evidence  suggests  that  IPT  training  and  participation  on 
effective  IPTs  can  help  in  overcoming  this  resistance — primarily  through  changes  in 
underlying  outcome  beliefs.  The  next  section  examines  IPTs. 
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Briefing  Overview 


•  Overall  progress  of  reform 

•  Progress  in  reforming  miispecs  and  standards 

•  Impiementation  of  integrated  product  teams  (iPTs) 

•  Movement  toward  industry  partnerships 

•  Recommendations  for  improving  reform 
implementation 


Arroyo  center 


RAND 


Given  that  greater  use  of  IPTs  is  a  major  emphasis  of  acquisition  reform  efforts  and  IPTs  have 
proven  to  be  useful  in  facilitating  support  for  milspec  and  standard  reform,  this  section 
examines  work  force  beliefs  about  IPTs  and  their  relationship  to  milspec  and  standard  reform. 
Specifically,  this  section  will  examine  the  IPT-related  outcome  beliefs  of  the  acquisition  work 
force;  the  relationship  between  these  beliefs  and  work  force  views  on  the  ability  of  IPTs  to 
facilitate  milspec  and  standard  reform;  and  the  role  of  various  IPT  training  strategies  in 
changing  IPT-related  outcome  beliefs.  In  conclusion,  we  will  comment  on  the  “conventional 
wisdom”  about  what  makes  IPTs  successful  in  the  private  sector  and  provide  recommendations 
on  how  to  better  implement  teams  in  the  Army  context. 

To  improve  coordination  among  the  many  players  in  defense  acquisition  and  to  ultimately 
streamline  the  process,  the  “Perry  Initiatives”  identified  the  need  for  greater  use  of  Integrated 
Product  Development  Teams  (IPDTs)  or  IPTs.  Teams  have  become  a  major  focus  within  DoD 
and  the  services.  They  are  designed  to  minimize  “stovepiping”  of  functional  duties,  increase 
communication — both  laterally  and  vertically — and  reduce  oversight  and  system  delays  to 
streamline  the  process. 

Unfortunately,  the  acquisition  work  force  lacks  a  common  and  well-understood  definition  of 
IPTs.  The  definitional  problem  may  stem  from  the  fact  that  teams  can  be  and  are  expected  to 
operate  at  various  levels  within  the  acquisition  system.  Within  DoD,  teams  are  thought  to 
operate  at  basically  three  levels: 

(1)  Overarching  IPT:  This  team  is  made  up  of  DoD  and  service  officials  from  the  Program 
Manager  (PM)  level  and  above— principally  PMs,  Program  Executive  Officers  (PEOs),  service 
officials,  and  DoD  officials. 

(2)  Program-level  IPT:  This  team  is  made  up  of  those  active  in  the  management  of  a  program. 

It  is  essentially  a  multifunctional  team  that  spans  the  current  matrix  management  system. 

(3)  Contractor  IPT:  This  team  is  made  up  of  PM  personnel  and  contractor  personnel.  It  is  the 
most  product  oriented  of  the  three  types  of  teams  (the  others  being  more  process  oriented). 

The  analysis  in  this  section  focuses  on  a  general  “IPT  concepf  most  accurately  described  as 
the  program-level  and  contractor  IPTs  discussed  above. 
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Beliefs  About  IPTs  Are  Largely  Positive 


Factor  affected 
by  IPTs 

Respondents 
expecting  an 
increase 

Respondents 
expecting  a 
decrease 

Development  costs 

13% 

47% 

Product  quality 

57% 

9% 

User  satisfaction 

62% 

6% 

Communication 

78% 

4% 

Oversight 

24% 

34% 

Arroyo  center 

RAND 

In  general,  beliefs  about  IPTs  are  largely  positive.  In  the  survey,  respondents  were  asked 
whether  they  believed  that  IPTs  would  result  in  an  increase  or  decrease  in  several  key 
program  outcomes  (development  costs,  product  quality,  user  satisfaction)  or  acquisition 
process  factors  (communication,  oversight)— these  are  identified  in  the  leftmost  column. 
Based  on  the  survey,  it  appears  that  respondents  expect  IPTs  to  promote  improvements  in 
development  costs,  product  quality,  user  satisfaction,  and  communication.  On  the  other 
hand,  respondents  were  not  as  optimistic  about  effects  on  oversight,  with  24  percent 
versus  34  percent  expecting  an  increase  rather  than  a  decrease.  These  figures  also  imply 
that  42  percent  of  those  sunreyed  believe  that  IPTs  will  not  affect  the  level  of  oversight. 
During  many  of  our  site  visits  we  heard  several  anecdotes  that  supported  this  belief. 
Teaming  approaches  are  predicated  on  a  free  flow  of  information,  with  participants  letting 
go  of  their  traditional  adversarial  roles.  Some  of  our  interviewees  believed  that  external 
auditors  have  a  difficult  time  working  in  an  environment  that  stresses  coiiaborative 
solutions  rather  than  reactive  sanctions. 
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Why  Do  Respondents  Believe  That  “IPTs  will 
make  it  easier  to  eliminate  miispecs”? 


Using  IPTs  will 
increase... 

Effect  on 
Ability  to 
Reform 

Job  satisfaction 

+ 

Development  costs 

— 

Life-cycle 

costs 

— 

Work  required 

— 

Promotion 

opportunities 

+ 

Oversight 

— 

Arroyo  center 


1  Effect  on 

Using  IPTs  will 

Ability  to 

increase... 

Reform 

Innovation 

+ 

Program  delays 

Other  Factors... 

1  Effect  on 
Ability  to 
Reform 

Poor  data  on  past  perf. 

IPT  experience 

■ 

(effective) 

+ 

Received  IPT  training 

+ 

RAND 


Rocalling  somG  of  our  earliGr  rGsults,  it  was  dGtGrminGd  that  IPT  GxpGriGncG  and  training  ara 
rslatGd  to  graatar  support  for  milspac  and  standard  raform  and  positiva  outcoma  baliafs  with 
ragard  to  tha  affacts  of  milspac  and  standard  reform.  This  chart  reports  the  results  of  an 
ordered  logistic  regression  conducted  to  explain  the  observed  distribution  of  responses  to 
the  following  statement:  “IPTs  make  it  easier  to  eliminate  miispecs  and  standards."  This 
model  predicts  the  response  distribution  (from  “strong  agreement”  to  “strong  disagreement”) 
for  this  statement  based  on  the  specific  outcome  beliefs  of  survey  respondents.!  Basically, 
this  model  identifies  those  IPT-related  outcome  beliefs  that  are  important  in  shaping 
employee  beliefs  with  regard  to  the  effect  of  IPTs  on  their  willingness  to  implement  milspec 
and  standard  reform. 

The  beliefs  reported  in  this  chart  are  listed  in  order  of  importance  in  explaining  the  response 
distribution  to  the  statement.  Once  again,  a  plus  sign  (+)  indicates  a  positive  correlation 
between  an  outcome  belief/factor  and  the  level  of  agreement  with  the  statement  (the  greater 
the  agreement  with  an  outcome  belief,  the  greater  the  agreement  that  IPTs  will  make  it 
easier  to  eliminate  miispecs  and  standards).  Likewise,  a  minus  sign  (-)  indicates  a  negative 
relationship  (the  greater  the  agreement  with  an  outcome  belief,  the  lower  the  agreement 
with  the  statement).  For  example,  beliefs  about  the  effect  of  IPTs  on  job  satisfaction  were 
the  most  important  determinant  of  agreement  with  the  statement.  In  this  case,  the  greater 
the  belief  of  a  respondent  that  IPTs  will  increase  job  satisfaction,  the  greater  his  agreement 
that  IPTs  will  make  it  easier  to  eliminate  miispecs  and  standards. 


^The  basic  form  of  the  model  employed  here  is 

(Response  Distribution)  =  /  (IPT-related  outcome  beliefs,  IPT  participation,  and 

acquisition  process  beliefs). 

While  the  primary  independent  variables  are  IPT-related  outcome  beliefs,  several  other  categories  of 

independent  variables  were  considered,  including  IPT  experience  and  “acquisition  process  beliefs” _ 

such  as  respondent  beliefs  on  the  adequacy  of  past  performance  data— all  of  which  were 
hypothesized  to  affect  beliefs  about  IPTs. 
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In  addition  to  beliefs  about  job  satisfaction,  those  respondents  who  believe  IPTs  will 
increase  promotion  opportunities  and  innovation  within  the  acquisition  system  are  more 
likely  to  be  positive  about  the  efficacy  of  IPTs  in  eliminating  milspecs  and  standards. 

Negative  outcome  beliefs  are  inversely  correlated  with  agreement  with  the  statement. 
Specifically,  respondents  who  believe  that  using  IPTs  will  increase  development  costs,  life- 
cycle  costs,  work  required,  oversight,  and  program  delays  are  more  likely  to  find  IPTs  less 
than  effective  in  facilitating  milspec  and  standard  reform.  An  implication  of  this  result  is  that 
respondents  who  do  not  believe  that  IPTs  will  result  in  these  negative  outcomes  are  more 
likely  to  find  IPTs  useful  in  facilitating  milspec  and  standard  reform. 

Going  beyond  IPT-related  outcome  beliefs,  three  “other  factors”  appear  important  in 
predicting  the  response  distribution  for  this  statement.  Respondents  who  believe  that  the 
Army’s  past  performance  data  are  “poor”  are  less  likely  to  believe  that  IPTs  are  effective  in 
facilitating  reform.  On  the  other  hand,  respondents  who  have  experience  on  an  “effective” 
IPT  (an  IPT  with  a  charter)  and  who  have  received  IPT  training  are  much  more  likely  to  find 
IPTs  effective  in  facilitating  milspec  and  standard  reform.  In  fact,  although  not  reported 
explicitly  in  this  chart,  based  on  our  analysis  it  appears  that  serving  on  an  IPT  without  a 
charter  made  a  negative  contribution  to  employee  beliefs  about  the  efficacy  of  IPTs. 

Finally,  while  not  identified  explicitly  in  the  chart,  career  group  remains  an  important 
determinant  of  beliefs.  In  this  case,  the  general  pattern  holds,  with  QA  personnel  being  the 
most  negative  and  least  likely  to  view  IPTs  as  facilitating  milspec  and  standard  reform,  and 
military,  logistics,  and  contracting  personnel  being  generally  more  positive. 
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Effectiveness  of  IPT  Training  Varies 


Average  increase  in  respondents  who  “strongly  agree” 


External 

training 

Internal 

training 

Defense 

Acquisition 

University 

Contractor- 

provided 

training 

IPTs  Improve  work 
processes  (8  items) 

+60% 

+50% 

+30% 

+10% 

IPTs  improve  program 
outcomes  (1 5  items) 

+50% 

+20% 

+20% 

+15% 

Arroyo  center 

RAND 

Beliefs  about  the  effects  of  IPTs  were  greatly  affected  by  participation  in  various  forms  of 
training.  The  above  chart  summarizes  the  results  of  regression  analyses  of  answers  to 
questions  about  improvements  in  work  processes  (8  items)  and  in  program  outcomes  (15 
items).  Controlling  for  several  other  factors  (such  as  career  group,  tenure,  etc.),  the 
summary  provides  a  measure  of  the  average  percentage  increase  in  the  number  of 
respondents  who,  as  a  result  of  training,  “strongly  agree”  that  IPTs  will  improve  the  work 
processes  or  program  outcomes. i 

External  training  is  training  provided  by  an  outside  consultant,  generally  at  the  request  of 
the  local  command  or  program  office.  These  consultants  include  local  community  colleges 
and  private  firms.  This  type  of  IPT  training  was  the  most  effective,  resulting  in  an  increase 
of  60  percent  and  50  percent  in  the  number  of  individuals  reporting  that  they  “strongly 
agree”  that  IPTs  improve  work  processes  and  program  outcomes,  respectively. 

Internal  training,  generally  provided  by  program  management,  is  also  quite  effective  in 
increasing  beliefe  about  work  processes.  It  was  substantially  less  effective  than  external 
training  in  affecting  beliefs  about  the  effects  of  IPTs  on  program  outcomes. 

Training  provided  by  DAU  was  somewhat  less  effective  than  internal  training  with  regard  to 
beliefs  about  work  processes,  and  had  equal  effectiveness  on  beliefs  about  program 
outcomes.  While  less  effective  than  external  training,  both  internal  and  DAU  IPT  training 
resulted  in  positive  shifts  in  the  distribution  of  beliefs. 


^The  eight  items  relating  to  work  processes  included  factors  such  as  general  perspectives  on 
employee  understanding  of  IPTs  and  their  role  on  teams,  as  well  as  items  measuring  the  effect  of 
IPTs  on  work  processes  (3  items),  decisionmaking,  ease  of  adopting  milspec  and  standard  reform, 
and  ease  in  adopting  best-value  contracting  methods.  The  15  items  relating  to  program  outcomes 
assessed  employee  beliefs  about  the  effect  of  IPTs  oni  development  costs,  product  quality,  user 
satisfaction,  concept  to  product  time,  rework,  innovation,  communication,  job  satisfaction,  program 
delays,  oversight,  conflict  between  PM  and  functional  directors,  conflict  between  government  and 
user,  life-cycle  costs,  workload,  and  advancement  opportunities.  Some  of  these  outcomes  are  not 
strictly  programmatic,  but  do  relate  to  program  success. 
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Training  provided  by  defense  contractors  (the  same  firms  providing  the  Army  with  R&D, 
production,  etc.)  was  the  least  effective.  However,  in  these  cases,  large  uncertainty  in  the 
estimates  (primarily  because  of  the  small  sample  size  for  this  category)  makes  it  difficult  to 
come  to  definitive  conclusions. 
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While  it  has  already  been  pointed  out  that  only  about  52  percent  of  the  surveyed  work  force 
had  participated  in  one  or  more  forms  of  IPT  training,  it  is  worth  noting  that  participation  in 
IPT  training  does  vary  by  career  group. 

By  and  large,  military  personnel  have  received  the  most  IPT  training— only  38  percent  have 
received  no  IPT  training.  Contracting  personnei,  on  the  other  hand,  have  the  lowest  level 
of  IPT  training— 68  percent  have  received  no  IPT  training.  This  corresponds  with 
information  gained  in  our  interviews  of  acquisition  personnel.  In  general,  military  personnel 
are  in  program  management  roles.  IPTs  are  commonly  viewed  as  a  program  management 
improvement,  and  these  personnel  are  more  heavily  trained.  Contracting  personnel,  on  the 
other  hand,  are  commonly  viewed  as  independent  of  the  program  office  and  are  not  as 
likely  to  be  involved  in  PM-directed  IPT  training  activities. 

Internal  training  appears  to  be  the  most  popular  form  of  IPT  training,  with  between  18  and 
25  percent  of  each  career  group  having  participated  in  it.  The  “most  effective”  form  of 
training — externally  provided  training — is  also  well  subscribed,  with  between  12  and  25 
percent  of  each  career  group  participating.  While  a  substantial  number  of  respondents 
have  experienced  some  form  of  IPT  training,  a  large  majority  have  not  received  the  two 
most  effective  forms.  If  IPT  training  helps  in  making  employees  more  willing  to  eliminate 
milspecs  and  standards,  intensified  training  should  be  considered. 
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Four  Keys  to  IPT  Success 

1 .  Used  only  when  appropriate 

•  when  integration  wiil  assist  in  meeting  objectives 

2.  Guided  by  clear  and  consistent  objectives  and  responsibilities 

•  well  defined  and  understood  task 

•  clear  and  consistent  responsibilities— team  charters 

3.  Emphasis  on  proper  staffing 

•  key  roles  and  functions  represented 

•  training  provided  to  all  members 

•  special  focus  on  team  leaders 

4.  Strengthened  by  organizational  support  systems 

•  performance  evaluation  systems 

•  training  programs 

•  empowerment/authority  to  team  leaders  and  members 

•  information  systems 

>\rroyo  center 

RAND 

The  importance  of  training  to  beliefs  and  perceptions  about  IPTs  is  not  necessarily  surprising. 
An  essential  element  of  successful  teams  is  an  adequately  trained  and  motivated  work  force. 
Based  on  both  our  review  of  the  literature  on  teaming  and  on  our  site  visits  to  various 
corporations,  we  were  able  to  identify  four  broad  “keys”  to  successful  IPTs.  The  keys  to 
success  are  described  in  the  chart  above. 

First,  IPTs  should  be  used  only  when  appropriate.  Rather  than  mandating  teams  broadly 
across  the  organization,  successful  organizations  employ  teams  carefully  and  only  when  they 
are  deemed  appropriate.  Basically,  teams  should  be  used  when  there  are  clear  advantages  to 
integrating  and  communicating  across  functions  or  disciplines  in  the  organization.  In  addition, 
teams  can  be  time  consuming  and  costly  (particularly  initially),  and  they  should  be  used  only 
when  there  are  sufficient  resources  (financial,  human,  and  time)  available. 

Second,  successful  IPTs  require  a  predefined  task  or  objective  around  which  team  members 
can  be  motivated.  In  order  to  meet  this  objective,  successful  teams  often  use  clear  and 
consistent  statements  of  responsibilities — sometimes  called  team  charters.  These  charters  lay 
out  the  team’s  objective,  the  plan  for  meeting  it,  and  the  responsibilities  of  team  members. 

Third,  organizations  associated  with  successful  IPTs  place  a  great  deal  of  emphasis  on 
properly  staffing  their  IPTs.  Properly  staffed  IPTs  include  all  key  actors  and  functions  involved 
in  meeting  the  team’s  objective.  In  addition,  these  organizations  provide  IPT  training  to 
members  in  order  to  improve  team  efficacy.  Moreover,  a  special  emphasis  is  placed  on  the 
selection  and  training  of  team  leaders. 

Fourth,  effective  IPTs  require  support  from  other  elements  of  the  organization.  Basically,  an 
IPT  cannot  function  “on  its  own”— organizational  support  systems  must  be  modified  to  enable 
and  facilitate  effective  teams.  The  organizational  support  systems  that  must  be  modified 
include  performance  evaluation  and  reward  systems  which  must  account  for  team,  as  well  as 
individual,  performance:  training  programs  which  need  to  address  communication  and  teaming 
skills;  empowerment/authority  guidelines,  to  provide  team  members  necessary  autonomy  and 
decisionmaking  power;  and  information  systems  which  provide  for  effective  communication  and 
management. 
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Conclusion:  IPTs  Have  Great 
Untapped  Potential 

•  Successful  IPTs  require  attention  to  specific  considerations  and 
organizational  supports— especially  training  and  leadership 

•  Empirical  evidence  links  effective  IPTs  with  positive  attitudes  and 
preferred  behaviors 

-  Respondents  are  more  iikely  to  believe  IPTs  will  increase 
quality  and  user  satisfaction 

-  More  iikely  to  support  milspec  and  standards  reform 

•  Substantial  improvements  are  possible  in  Army  IPTs 

-  Currently,  only  50%  of  respondents  have  been  on  an  IPT 

-  Only  50%  of  respondents  have  received  training 

-  There  is  no  common  definition  or  understanding  of  IPTs 

-  40%  of  respondents  do  not  understand  their  role  on  a  team 

-  Only  30%  of  respondents  indicate  their  team  has  a  written 
charter;  38%  don’t  know  whether  or  not  they  have  a  charter 

Arroyo  center 


As  the  IPT  literature  maintains  and  our  interviews  with  private-sector  firms  confirmed,  IPTs 
are  most  effective  when  they  are 

•  used  appropriately, 

•  guided  by  written  charters  and/or  mission  statements, 

•  staffed  properly,  and 

•  strengthened  by  necessary  organizational  support  systems. 

Among  support  systems,  training  is  essential,  especially  for  team  leaders.  Moreover,  the 
experience  of  Army  programs  implementing  IPTs  suggests  that  training  is  most  effective 
when  it  is  locally  tailored  (internal  training  by  a  PM  or  Buying  Command)  and/or  provided 
by  professionals  (external  training).  The  most  effective  IPTs  and  IPT  training  we  observed 
was  conducted  by  highly  motivated  and  trained  program  managers. 

Our  analysis  also  indicates  that  experience  with  an  “effective”  IPT  (in  our  definition,  one 
with  a  charter)  can  be  linked  to  positive  attitudes  and  preferred  behaviors  (such  as 
implementing  milspec  and  standard  reform).  For  example,  respondents  who  have  served 
on  such  IPTs  believe  that  they  will  increase  quality  and  user  satisfaction.  In  addition,  those 
with  experience  on  an  effective  IPT  are  more  likely  to  support  milspec  and  standard  reform. 

While  the  evidence  on  IPTs  and  IPT  training  in  the  Army  is  largely  positive,  substantial 
improvements  are  possible.  Currently,  only  half  of  all  respondents  participate  on  IPTs  and 
half  of  all  respondents  have  received  IPT  training.  There  is  no  common  definition  or 
understanding  of  IPTs  within  the  acquisition  work  force,  and  40  percent  admit  that  they  do 
not  understand  their  role  on  a  team.  Only  30  percent  indicate  that  their  team  has  a  written 
charter — qualifying  it  as  an  effective  IPT  in  our  analysis — and  almost  40  percent  do  not 
even  know  whether  the  teams  they  have  served  on  have  had  or  have  a  charter.  IPT 
participation  under  such  circumstances  contributes  little  to  acquisition  reform  and,  in 
contrast,  may  actually  work  against  the  ultimate  objectives. 
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Recommendations:  Establishing  Effective  iPTs 


•  Use  team  charters  and  training  to  ensure  role  clarity  and  common 
understanding  of  responsibilities  and  team  goals 

•  Tailor  team  size  and  structure  to  program  and  command  needs 

-  Small  programs,  “basket”  PMs  require  different  teams 

-  Tailor  training  to  needs  of  individual  programs  and  commands 

•  Emphasize  leadership  in  overall  teaming  effort  and  on  teams 

-  Continue  to  use  PMs  to  implement  teams 

-  Train  team  leaders 

•  Explore  team-based  evaluations  and  rewards 

-  Will  be  hard  with  different  military  and  civilian  systems/rules 

-  Consider  continued  use  of  nonmonetary  rewards 
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Based  on  our  review  of  the  literature  on  IPTs,  our  site  visits  with  private-sector  firms,  and 
our  interviews  with  Army  personnel  attempting  to  implement  IPTs,  it  appears  that  effective 
IPTs  require  a  common  team  concept  (or  set  of  concepts)  and  definition.  The  organization 
and  its  members  must  understand  what  teams  are  and  how,  and  when,  they  are 
appropriate.  One  can  bolster  common  definitions  and  role  clarity  through  extensive  team 
training  programs.  On  our  site  visits  to  private-sector  firms,  we  were  struck  by  the  fact  that 
most  successful  organizations  have  dramatically  increased  their  budgets  for  training,  even 
as  they  downsize  other  functions  and  departments. 

In  addition,  common  definitions  and  objectives  are  needed  at  the  individual  team  level. 
Team  charters  can  be  used  to  ensure  role  clarity  and  a  common  understanding  of 
participant  responsibilities  and  team  objectives.  The  use  and  importance  of  charters  can 
also  be  emphasized  in  IPT-related  training. 

While  a  common  conception  of  what  IPTs  are  is  desirable,  there  is  no  standard  blueprint. 
Instead,  teams  need  to  be  tailored  (in  terms  of  size  and  structure)  to  suit  particular  program 
and  managerial  needs.  For  example,  small  programs  and/or  “basket”  PMs  have  different 
requirements  than  larger  ACAT  I  programs.  The  program  portfolios  of  basket  PMs  are 
often  diverse,  with  a  relatively  small  staff  spread  across  a  number  of  programs.  Applying  a 
standard  IPT  template— more  than  likely  developed  for  larger,  higher-profile  ACAT  I  and  II 
level  programs— may  not  make  sense  for  these  organizations.  In  addition,  as  noted  earlier, 
our  analysis  suggests  that  attitudes  and  beliefs  about  IPTs  are  best  shaped  through 
localized  or  program-specific  training  provided  by  external  sources  or  the  PM. 

Emphasis  must  be  placed  on  leadership — both  in  the  broader  effort  to  implement  teams 
Army-wide  and  on  individual  teams.  As  we  noted  earlier,  in  our  interviews  with  Army 
personnel,  the  most  effective  teams  were  those  that  were  strongly  supported  and 
sponsored  by  the  PM.  In  addition,  these  teams  placed  a  great  deal  of  emphasis  on  the 
quality  of  their  team  leaders. 

Finally,  the  Army  must  explore  team-based  evaluation  and  reward  systems.  In  many  of  the 
Army  programs  and  commands  we  visited,  new  IPTs  were  simply  placed  within  existing 
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management  systems.  Employees  were  evaluated  by  their  traditional  “reviewers,”  and 
team-based  evaluations  were  rare.  In  addition,  team  responsibilities  were  often  simply 
added  to  an  employee’s  current  duties— giving  the  impression  of  added  burden  without 
linkage  to  an  employee’s  performance  evaluation.  In  addition  to  modifying  the  evaluation 
system,  attention  should  be  given  to  the  rewards  provided  by  that  system.  However,  we 
realize  team-based  monetary  rewards  will  be  difficult  to  implement  in  the  Army  context. 
Different  rules  and  pay  systems  for  military  and  civilian  employees  will  make  it  hard  to 
reward  team  members  equally  for  performance.  Rewards  can,  however,  be  nonmonetary. 
The  Army  should  consider  continued  use  of  medals,  other  forms  of  peer  recognition,  and 
“time  off’/leave  to  encourage  team  behavior.  In  addition,  budgetary  rewards— monetary 
rewards  accruing  to  the  program  or  the  activities  of  the  team — ^should  also  be  considered. 
Simply  allocating  more  money  in  support  of  a  team’s  objective  may  provide  incentive 
enough  for  team  cooperation  and  initiative. 
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Briefing  Overview 


Overall  progress  of  reform 

Progress  in  reforming  milspecs  and  standards 

Implementation  of  integrated  product  teams  (IPTs) 

Movement  toward  industry  partnerships 

Recommendations  for  improving  reform 
implementation 


Arroyo  center 


RAND 


As  noted  earlier,  a  third  key  element  of  the  Perry  Initiatives  is  a  new  emphasis  on  the 
creation  of  “buyer-supplier  partnerships.”  In  an  effort  to  make  defense  contracting  more 
appealing  (to  both  current  contractors  and  commercial  contractors),  DoD  and  the  Army  are 
attempting  to  improve  relationships  with  their  supplier  base.  These  improvements  are 
modeled  along  those  taking  place  in  private  industry.  In  private  industry,  increased 
emphasis  has  been  placed  on  long-term  stable  relationships  between  buyers  and 
suppliers.  Major  commercial  firms— some  of  which  were  interviewed  for  this  study— are 
developing  closer,  more  cooperative  relationships  with  their  suppliers.  Common 
components  of  these  relationships  are  buyer-supplier  IPTs,  best-value  contracting,  long¬ 
term  contracts,  preferred  supplier  status,  and  mutual  trust. 

DoD  and  the  Army  are  attempting  to  adopt  similar  relationships  with  their  suppliers.  As 
noted  earlier,  IPTs  are  an  essential  element  in  creating  better  buyer-supplier  relationships. 
The  increased  communication  and  cooperation  inherent  in  the  teaming  process  helps  to 
create  and  foster  better,  longer-term  relationships  between  buyers  and  suppliers.  In 
addition,  better  relationships  with  suppliers  will  be  essential  to  the  success  of  milspec  and 
standard  reform.  In  most  cases.  Army  program  offices  are  relying  more  on  the  contractor 
to  propose  alternative  performance-based  requirements  for  traditional  milspecs  and 
standards.  Without  traditional  milspecs  to  rely  on,  the  Army  must  have  better 
communication  with  contractors  and  confidence  in  the  capabilities  of  its  suppiiers.  For 
example,  without  the  MILQ-9858  quality  standard.  Army  QA  personnel  and  PMs  must  now 
rely  on  ISO  9000  or  similar  quality  certifications  and/or  jointly  developed  program-specific 
quality  plans. 

This  final  section  of  the  briefing  relies  largely  on  interview  and  site  visit  data  to  describe  the 
“new  model”  of  buyer-supplier  relations,  comment  on  the  extent  to  which  “partnerships”  are 
being  attained  between  the  Army  and  its  suppliers,  and  provide  some  generai  conclusions 
and  recommendations  on  how  the  partnering  process  could  be  improved  in  the  Army.  To 
conclude  this  section,  we  focus  on  one  particularly  promising  program  within  the  Army 
Materiel  Command  (AMC)— the  CP^  Program— which  can  serve  as  the  foundation  for 
better  buyer-supplier  relationships  for  the  Army. 
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Workforce  Views  of  Partnerships 

Most  expect  positive  outcomes  (agree  vs.  disagree) 

•  Increase  quaiity  (60%  vs.  10%) 

•  Decrease  program  costs  (48%  vs.  17%) 

•  Increase  trust  (70%  vs.  4%) 

•  Increase  cooperation  (74%  vs.  4%) 

Experience  faiis  short  of  potentiai 

•  Only  30%  report  better  cooperation 

•  66%  cite  need  for  better  communications  systems 

•  45%  indicate  past  performance  data  are  poor 

Regression  analysis  identifies  several  possible  levers 

•  Effective  IPTs  and  IPT  training  improve  beliefs 

•  Past  performance  data  strengthen  beliefs  and  can  be 
linked  to  higher  support  for  all  reforms 


Arroyo  center 


RAND 


Before  discussing  the  specifics  of  the  “new/  model”  for  buyer-supplier  partnerships  and  the 
degree  to  w/hich  the  Army  and  its  contractors  have  “partnered,”  it  is  useful  to  examine  the 
survey  data  and  the  level  of  support  for  partnerships  within  the  acquisition  work  force. 

Perhaps  somewhat  surprisingly  (given  the  history  of  adversarial  relationships  between  the 
government  and  its  contractors),  work  force  views  on  industry  partnerships  were  quite 
positive.  Most  employees  expect  positive  outcomes  to  result  from  better  partnerships.  For 
example,  60  percent  expect  an  increase  in  product  quality  (1 0  percent  disagree,  with  the 
remaining  30  percent  staying  neutral).  Program  costs  are  expected  to  decrease  and 
relationships  with  industry  are  expected  to  improve.  Most  expect  trust  and  cooperation  to 
be  enhanced. 

On  the  other  hand,  experience  to  date  may  be  falling  short  of  expectations.  Although  74 
percent  anticipate  improved  cooperation  with  industry,  only  30  percent  actually  report 
better  cooperation.  One  common  problem  cited  in  our  interviews  with  Army  personnel  was 
the  absence  of  adequate  systems  for  communications  (e.g.,  compatible  computers).  This 
was  confirmed  by  the  survey.  Another  problem— identified  in  our  interviews  and  confirmed 
by  the  survey  data — is  a  lack  of  confidence  in  past  performance  data  (upon  which  a 
partner  can  be  evaluated).  Almost  half  (45  percent)  of  those  surveyed  believe  that  past 
performance  data  are  “poor”  (only  27  percent  find  the  data  “good”  and  28  percent  had  no 
opinion). 

Additional  regression  analysis  identified  several  possible  policy  levers  affecting  employee 
perceptions  of  partnerships.  Again,  effective  IPTs  and  IPT  training  improved  beliefs  about 
the  benefits  of  partnerships — corresponding  to  their  central  role  in  “effective”  partnerships. 
Finally,  past  performance  data,  when  viewed  as  adequate,  could  be  linked  with  more 
optimism  about  the  effect  of  partnerships  on  program  outcomes  and,  therefore,  support  for 
partnerships.  To  the  extent  that  data  can  be  used  to  evaluate  competitive  bidders  on  future 
contracts,  firms  will  have  stronger  incentives  to  perform  well  on  current  contracts.  When 
such  incentives  are  in  place,  less  oversight  (and  more  trust)  and  true  partnerships  are 
possible. 
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Buyer-Supplier  Relations  in  the 

Private  Sector  Are  Changing 

Old  model 

New  model 

Procurement  a  low  priority 

Procurement  part  of  overall  strategy 

Low  price 

Best  value 

Short-term  contracts 

Long-term  relationships 

Many  suppliers 

Preferred  suppliers 

Market  relationship 

Cooperation 

Design  preceding  purchase 

Suppliers  involved  in  development 

Emphasis  on  past  performance 

Greater  emphasis  on  past  performance 

Quality  test  on  delivery 

Demonstrated  quality  process 

Adversarial  relationship 

Mutual  trust 

Arroyo  center 

RAND 

This  slide  summarizes  what  we  mean  by  the  “new  partnership  model.”  Based  on  a  survey 
of  the  literature  dealing  with  partnerships,  it  becomes  evident  that  the  “new  model” 
represents  a  serious  departure  from  the  old.  Basically,  the  “new  model”  places  increased 
emphasis  on  the  role  of  suppliers  and  their  products.  It  focuses  on  quality  and  best-value 
contracts  as  the  foundation  for  long-term,  stable,  cooperative,  and  trusting  relationships 
with  a  smaller  number  of  preferred  suppliers. 

In  general,  many  aspects  of  the  “new  model” — such  as  best  value,  long-term  relationships, 
and  demonstrated  quality  processes — are  elements  of  current  Army  acquisition  reform 
efforts.  Through  our  site  visits  with  Army  program  offices  and  their  prime  contractors  we 
were  able  to  assess  the  extent  to  which  progress  had  been  made  in  reaching  this  “new 
model.” 
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Progress  Toward  Partnerships 

Program  /  Contractor 
Contract  for  design/systems 
Concentration  on  a  few  key  suppliers 
Long-term  relationship 
Long-term  contracts/commitments 
Best-value  source  selection 
Joint  IPT  for  RFP/proposal 
New  approach  to  quality  (prooinsp) 

Greater  sharing  of  information 
Mutual  trust 
IPTs  for  program  management 
Continuous  improvement/cost  reduction 
Greater  end-user  involvement 
Suppliers  rating  system 

=  substantial  progress 

Arroyo  center 


This  slide  summarizes  our  findings  from  eight  of  our  visits  to  defense  contractors  and 
discussions  with  the  matched  sample  of  Army  programs  identified  earlier.  Therefore  each 
letter  represents  a  given  program  office-prime  contractor  pairing.  For  reasons  of 
anonymity,  the  pairings  are  labeled  “A”  through  “H.” 

For  each  pairing,  we  evaluated  the  extent  to  which  the  interviewees  gave  evidence  of 
progress  in  terms  of  the  partnership  dimensions  listed  on  the  left  of  the  chart.  We 
concluded  that  the  program  office  and  the  contractor  had  made  “substantial,”  “some,”  or 
“no"  progress  in  achieving  the  private  industry— or  “new  model”  standard  of  partnering. 

For  example,  defense  contracting  has  always  involved  contracts  for  design  or  system 
integration,  and  long-term  relationships  with  a  small  set  of  key  suppliers.  By  these  criteria, 
defense  contractor-government  relationships  look  like  partnerships  in  the  private  sector. 
Also  encouraging  is  the  degree  of  progress  in  application  of  best-value  source  selection, 
the  use  of  joint  IPTs  for  the  RFP/proposal,  the  use  of  new  approaches  to  quality,  and 
greater  end-user  involvement — with  at  least  some  progress  being  made  for  five  or  more  of 
the  eight  pairings  in  each  case. 

By  other  dimensions,  much  less  progress  has  been  made.  Pairings  D  through  H  exhibit 
some  or  no  progress  across  most  of  the  remaining  dimensions.  These  include  long-term 
contracts,  greater  sharing  of  information,  mutual  trust,  government-contractor  IPTs, 
continuous  improvement/cost  reduction,  and  suppliers  rating  systems. 

Examination  of  the  chart  suggests  that  pairings  A,  B,  and  C  have  been  most  successful  in 
moving  toward  “new  model”  style  buyer-supplier  partnerships.  A  key  question  is;  What  is 
different  about  these  pairings  that  may  indicate  why  they  are  relatively  more  successful? 
Without  revealing  the  identity  of  these  programs  or  contractors,  three  key  factors  seem  to 
help  explain  their  relative  success  in  moving  toward  closer  relations:  (1)  at  least  two  of  the 
three  programs  were  under  strong  budget  pressure— giving  both  the  program  and  the 
contractor  incentives  to  cooperate  in  order  to  “save”  the  system;  (2)  all  three  programs 
were  typified  by  strong  leadership  from  the  PM,  especially  regarding  reform 
implementation;  and  (3)  each  program  had  long-term  relationships  with  its  prime 
contractor,  assisting  cooperative  efforts. 
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Conclusion:  Significant  Barriers  to 
Partnerships  Exist 

Desired 

outcome 

Barriers 

Mutual  trust 

•  Independent  oversight 

•  Adversarial  history 

Supplier  ratings 

•  inadequate  past 
performance  data 

•  Lessons  not  shared 

New  approach  to 
quality 

•  QA  resistance 

•  External  oversight 

Greater  end-user 
involvement 

•  infiexible  ORD 

•  Layers  of  decision 
making 

Arroyo  center 

RAND 

In  this  chart,  we  isolate  a  few  of  the  most  important  elements— or  desired  outcomes — of 
effective  partnerships  and  identify  the  key  barriers  to  their  realization  in  the  Army  context. 
For  example,  building  greater  mutual  trust  between  the  Army  and  its  contractors  is 
hampered  by  the  existence  of  independent  oversight  functions  (outside  the  Army’s  control, 
or  outside  the  control  of  Army  acquisition  officials)  as  well  as  a  history  of  adversarial 
relationships. 

Movement  toward  supplier  ratings  systems  in  the  Army  has  been  hampered  by  poor  past 
performance  data.  Currently,  the  requisite  data  do  not  exist,  nor  are  lessons  learned 
shared  across  programs  (within  the  Army  or  across  services).  A  lack  of  data  makes  it 
difficult  for  the  Army  to  fairly  evaluate  “good”  partners  or  punish  “poor”  partners  from  the 
past. 

New  approaches  to  quality,  emphasizing  continuous  process  improvement  and  certification 
rather  than  testing  (such  as  the  ISO  9000  certification  process)  are  central  to  the  new 
buyer-supplier  paradigm.  In  our  interviews  with  Army  personnel  it  was  commonly  noted 
that  resistance  from  the  QA  function  (also  confirmed  by  survey  data)  and  the  external 
oversight  and  testing  camps  were  major  impediments  to  movement  toward  partnerships. 
Obviously  a  new  definition  for  quality — one  that  focuses  on  certified  quality  processes 
rather  than  extensive  product  testing— will  be  hard  to  implement  in  a  system  that  relied  on 
the  specific  quality-related  miispecs  and  standards  (MILQ-9858,  etc.). 

Finally,  the  new  paradigm  of  partnering  requires  greater  end-user  or  customer  involvement. 
Greater  user  involvement  is  often  cited  as  an  important  factor  in  determining  the  ultimate 
success  of  acquisition  reform.  Unfortunately,  data  from  our  interviews  suggest  that  user 
participation  in  acquisition  and  its  reform  varies  by  program  and  by  user.  In  our  interviews 
with  acquisition  personnel  it  was  often  stated  that  greater  user  participation  is  precluded  by 
inflexible  operational  requirements  (ORDs)  and  the  layers  of  bureaucracy  Involved  in 
changing  requirements. 
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Recommendations:  Gradually 

Address  Barriers 

Desired 

outcome  I  Barriers  |  Solutions 

Mutual  trust 

•  Independent  oversight 

•  Adversarial  history 

•  Build  gradually  via  IPTs, 
supplier  ratings,  and 
source  selection 

Supplier  ratings 

•  Inadequate  past 
performance  data 

•  Lessons  not  shared 

•  Expand  supplier 
certification  &  data  base 

•  Link  with  source 
selection 

New  approach  to 
quality 

*  QA  resistance 

•  External  oversight 

•  Scale  down 
inspectorate 

•  Retrain  for  new  role 

Greater  end-user 
involvement 

Arroyo  center 

•  Inflexible  ORD 

•  Layers  of  decision 
making 

•  User  IPT  participation 

•  Empowered  user 
representative 

RAND 

As  this  slide  indicates,  we  can  envision  several  steps  the  Army  can  take  to  help  address  the 
barriers  to  effective  partnerships. 

Developing  mutual  trust  will  naturally  be  a  gradual  process.  The  Army  should  consider  using 
government-contractor  IPTs  to  increase  the  flow  of  information  and  communication.  In 
addition,  the  development  of  a  supplier  rating  system  (likely  in  concert  with  industry)  will  help 
to  provide  government  officials  with  the  data  necessary  to  evaluate  and  select  partners. 

Such  supplier  rating  systems  are  a  key  element  of  effective  partnerships  in  the  private 
sector.  The  Army  should  maintain  efforts  to  develop  a  system  of  certifying  its  suppliers 
based  on  their  past  performance  and  other  considerations.  Such  a  system  can  then  be 
linked  to  source  selection  to  help  provide  stability  and  the  basis  for  long-term  relationships. 
As  we  will  discuss  in  the  next  two  slides,  the  foundation  for  such  a  supplier  certification  and 
rating  system  is  under  development  in  Army  Materiel  Command  (AMC)— the  Contractor 
Performance  Certification  Program  (CPs). 

If  the  primary  barriers  to  new  quality  approaches  are  coming  from  various  functional 
personnel  within  and  outside  the  Army,  it  may  be  advantageous  to  minimize  resistance  by 
scaling  down  “the  inspectorate”  and  retraining  employees.  “The  inspectorate”  refers  to  Army 
and  DoD  organizations  in  charge  of  “inspecting”  the  quality  of  products— such  as  the  QA 
function,  the  Defense  Contract  Management  Command  (DCMC),  the  Defense  Plant 
Representatives  Office  (DPRO),  and  the  Defense  Logistics  Agency  (DLA).  Consideration 
should  be  given  to  reducing  the  amount  of  product  inspection  and  increasing  the  amount  of 
process  certification.  In  all  likelihood,  such  a  change  in  activities  will  require  new  skills  and 
perspectives.  To  the  degree  possible,  efforts  should  be  made  to  retrain  QA  personnel  on 
new  quality  approaches,  such  as  ISO  9000  and  their  changing  role  in  the  acquisition  system. 

If  inflexible  ORDs  and  layers  of  decisionmaking  are  barriers  to  greater  user  involvement,  the 
Army  should  consider  placing  users  on  Army  IPTs  or  Army-contractor  IPTs.  These 
participants  must,  however ,  be  empowered  to  represent  the  user  and  to  modify  requirements 
as  needed.  This  may  be  a  difficult  task  without  the  full  cooperation  of  user  organizations. 


42 


Contractor  Performance  Certification 
Program  (CP  2)  is  Promising 

•  Possible  benefits  from  (CP^) 

-  Strengthens  Army-supplier  partnerships 

-  Reinforces  new  approach  to  quaiity 

-  Reduces  testing  and  oversight  costs 

-  May  offer  simultaneous  ISO  certification 

-  Emphasizes  continuous  improvement 

•  Implementation  is  uneven 

-  Progress  only  in  some  commands 

-  Limited  benefits  to  contractor 

Arroyo  center  RAND 


The  Contractor  Performance  Certification  Program  (CP2),  launched  originally  by  AMC 
in  the  late  1980s  and  then  substantially  revised  in  1995,  provides  a  potentially 
promising  movement  closer  to  best  practices  in  the  management  of  suppliers.  CP^ 
contains  ail  of  the  main  elements  of  ISO  9001 ’s  quality  certification  program, i  as  well 
as  more  stringent  criteria  in  areas  of  particular  importance  to  the  Army — e.g.,  design 
process  control  and  software  development.  It  also  adds  elements  not  included  in  ISO 
9001  that  are  related  to  the  public-sector  nature  of  the  procurement  process  and/or 
important  success  factors  for  the  development  of  complex  weapon  systems;  e.g., 
technical  risk  management,  IPTs,  warranty  performance,  environmental/hazardous 
waste  and  safety  procedures.  Thus,  the  CPs  initiative  is  similar  to  supplier  certification 
programs  at  industry  leaders  such  as  Boeing  and  Ford,  where  they  insist  on 
performance  above  and  beyond  the  ISO  standard  in  some  areas. 

Although  the  revised  CPs  is  still  in  the  early  implementation  phase,  experience  from 
commands  that  have  been  leaders  in  its  development  and  in  past  efforts  at  supplier 
certification  suggests  there  may  be  many  potential  benefits,  including: 

•  Helping  to  build  partnerships  with  primes  and  subcontractors  as  teams  of  Army 
quality  experts  do  precertification  audits  to  help  identify  areas  of  improvement; 
this  reinforces  the  new  approach  to  quality,  stressing  the  need  to  get  the  process 
right  the  first  time,  rather  than  inspecting  afterwards  for  quality  defects. 

•  Once  contractors  have  demonstrated  consistent  quality  processes  to  receive 
certification,  both  sides  can  achieve  cost  savings  by  eliminating  non-value-added 
tests  and  oversight. 


■'The  ISO  9000  series  is  the  International  Standard  Organization’s  standard  for  quality 
management  systems.  To  obtain  ISO  9000  certification,  an  organization  must  demonstrate  that 
it  has  ali  of  20  “key”  quality  management  components  in  place.  These  “key”  components  are 
considered  to  enable  an  organization  to  deliver  quality  products  or  services  on  a  consistent 
basis.  ISO  9000  is  rapidly  growing  in  popularity  as  multinational  corporations  insist  that  their 
suppliers  obtain  certification  in  order  to  standardize  quality  systems  on  a  global  basis. 
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•  Since  the  CP2  criteria  encompass  all  the  elements  of  ISO,  and  since  many  of  the 
Army’s  quality  inspectors  involved  in  the  initiative  have  received  ISO  training  and 
quality  auditor  status,  a  few  contractors  have  been  able  to  receive  ISO  certification  at 
the  same  time  as  CP2.  This  saves  both  sides  the  costs  and  time  required  for  separate 
certification  processes.2 


^Arrny  staff  have  been  reluctant  to  promote  this  benefit  for  fear  of  being  accused  of  unfair  competition 
by  private  ISO  auditors. 
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Improving  the  Potential  for 
Partnerships  by  Building  on  (CP 2) 

Build  on  (CP2) 

•  Link  more  closely  with  source  selection 

•  Extend  to  all  Army,  not  just  parts  of  AMC 

•  Expand  focus  of  ratings  to  overall  supplier 
performance 

•  Recognize  comparable  private-sector 
certifications 

•  Emphasize  capabilities  required  for  meeting 
military  performance  requirements 

Arroyo  center  rand 


One  of  the  critical  ways  in  which  the  Army  can  increase  the  incentives  for  contractors  to 
participate  in  CPz  is  to  link  the  supplier  certification  program  more  closely  to  source 
selection.  Currently,  the  reform  push  to  increase  the  use  of  past  performance  data  in 
source  selection  appears  to  be  progressing  quite  separately  from  CP2.  By  creating  a 
common  set  of  metrics  across  programs,  tracking  these  data  continuously,  and  using  them 
as  a  tool  for  both  program  management  and  a  key  criterion  for  future  source  selections,  the 
Army  can  move  much  closer  to  rewarding  good  performers  and  punishing  bad  ones — the 
foundation  of  private-sector  partnerships.  Indeed,  the  Air  Force’s  experience  with  supplier 
ratings  suggests  that  it  is  not  those  contractors  with  the  best  quality  and  delivery  records 
that  will  support  the  initiative.  Several  respondents  cited  the  Air  Force’s  CPARs  database 
as  a  potential  model,  although  this  appears  to  be  used  more  in  source  selection  than  in 
day-to-day  program  management,  with  information  not  made  public  among  the  supplier 
base. 

In  order  to  get  the  full  benefits  of  CP2,  the  Army  should  consider  extending  it  in  several 
ways:  first,  putting  pressure  on  those  elements  of  AMC  that  have  been  slow  in 
implementation  to  participate  more  actively  in  the  initiative;  second,  extending  it  throughout 
the  Army  through  a  clear  directive  from  the  Assistant  Secretary,  and  third,  expanding  the 
ratings  criteria  to  include  all  elements  of  contractor  performance — e.g.,  delivery  time, 
costs — and  not  just  quality. 

The  Army  could  also  consider  qualifying  certain  contractors  who  have  already  met  high 
commercial  quality  standards  on  closely  related  products — such  as  Boeing — without  a 
separate  CP^  certification  process.  It  is  important  to  recognize,  however,  that  meeting 
commercial  quality  standards  does  not  always  equate  with  the  capacity  to  meet  the 
military’s  often  more  stringent  environmental  and  performance  requirements.  One  of  the 
areas  where  CP2  could  add  real  value  is  in  identifying  contractors  that  have  demonstrated 
this  capability  and  helping  other  new  suppliers,  which  may  not  have  this  military 
experience,  to  understand  the  testing  and  other  capabilities  they  need  to  satisfy  the  Army’s 
needs. 
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A  Final  Word  on  Implementing 
Acquisition  Reform 

•  Successful  implementation  requires  alignment  among  a  large  number 
of  mechanisms 

•  Research  suggests  that  there  is  no  “silver  bullet”  for  facilitating  reform 

•  However,  several  promising  policy  levers  exist  that  would  collectively 
make  a  substantial  difference 

-  Break  down  functional  resistance  through  targeted  training, 
information,  and  communications  that  address  programmatic 
concerns 

-  Make  more  effective  use  of  IPTs  (internally  and  in  partnerships 
with  industry)  through  training,  mission  definition,  and  role 
clarification 

-  Promote  effective  partnerships  by  building  on  supplier 
certification  efforts,  developing  past  performance  data,  and 
linking  with  source  selection 

Arroyo  center 


While  we  have  provided  specific  conclusions  and  recommendations  related  to  facilitating 
and  improving  the  implementation  of  milspec  and  standard  reform,  IPTs,  and  buyer- 
supplier  partnerships,  several  important  and  general  conclusions  are  worth  reiterating. 

First,  successful  reform  requires  alignment  among  a  large  number  of  organizational 
support  mechanisms.  While  all  are  important  to  fostering  change  in  an  organization,  no 
single  factor  dominates.  For  example,  our  discussion  of  milspec  and  standard  reform 
emphasized  the  importance  of  focusing  training  efforts  on  the  programmatic  concerns  (with 
quality,  costs,  etc.)  of  those  most  opposed  to  reform.  In  addition  to  training,  the  discussion 
of  IPTs  identified  the  important  role  of  evaluation  systems,  empowerment,  and 
communication  and  information  systems  in  facilitating  reform. 

As  we  hope  the  analysis  and  conclusions  here  have  shown,  there  is  no  “silver  bullet”  for 
facilitating  reform.  A  great  deal  of  change  will  need  to  be  motivated  gradually  and  “on  the 
margin”  through  changes  in  many  variables — such  as  training,  human  resource  policies, 
and  new  information  systems  (past  performance  data). 

While  change  is  likely  to  occur  gradually  and  on  the  margin,  several  promising  policy  levers 
do  exist  that,  when  taken  together,  can  make  a  substantial  difference  in  our  estimation. 
First,  the  Army  should  do  more  to  break  down  functional  resistance  through  targeted 
training,  information,  and  communications  that  address  the  programmatic  issues  that 
impede  reform  (such  as  the  concern  for  quality).  Second,  the  Army  must  make  more 
effective  use  of  IPTs.  IPTs  are  an  important  element  of  acquisition  reform  because  they 
break  down  the  functional  barriers  that  impede  change— they  are  important  to  facilitating 
milspec  and  standard  reform  as  well  as  buyer-supplier  partnerships.  IPTs  can  be  bolstered 
through  training,  mission  definition,  and  role  clarification. 

Finally,  the  Army  can  promote  more  effective  partnerships  by  building  on  current  supplier 
certification  efforts,  developing  better  past  performance  data,  and  linking  these  data  with 
source  selection. 
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